The output of this step is a image where pixels with high probability of lanes are assigned max value of 255 and rest are suppressed to zero. This is achieved using a combination of operations that represent the criteria for a pixel being a lane. That is we use the fact that the lanes are usually white, part of an edge and mostly vertical. First the Sobel operator along x is applied to get the absolute binary threshold. Second, the gradient magnitude is thresholded. Third, the direction of the gradient is evaluated and finally, color thresholding is applied. The 'Edge' part is detected using (1 or (2 & 3)) and this is masked with the color threshold using (Edge & White Color).
The unnecessary part of the image is cropped off, leaving only the ground, by applying a Region-of-Interest mask. Then a perspective transform is applied to get a bird's eye view of the lanes to help calculations downstream.
Hopefully, the processing till now should leave an image where the pixels have a high probability of being lanes. We can then find a column wise histogram and classify the two peaks as left and right lanes. We then slice the image into uniform boxes, find mean of all the pixels in each box, classify it as a centroid and use that to realign the next sliding window. The location of the histogram peaks are chosen as a starting point for the sliding window search. Doing this easily solves the problem of finding curved lanes and improves upon Hough Transform in both speed and accuracy. The centroids from all the points are fit two a quadratic and the equation of the lane is found. This allows to extrapolate the lanes and also find the radius of curvature.
Below shows the visualization of all the previous steps. The equations of left and right lanes are calculated and the vehicle's relative position is calculated along with the radius of curvatures.
