How to check lane departure in OpenCV?

Question

I'm using OpenCV 3 in Python 2.7 on a Raspberry Pi 3. My project's aim is to build an autonomous lane departing robot that can detect the two lanes on its sides and continuously correct itself to remain within them. I want to achieve something like this project: https://www.youtube.com/watch?v=R_5XhnmDNz4

So far I've done the line detection part from the live video feed using both HoughLines and HoughLinesP. Here is a screenshot from my video feed and the outputs I'm getting so far:

Till now my logic for detecting if the robot is going left or right is based on the (rho,theta) output of the HoughLines function. What I want to achieve is a more robust way of tracking how the robot is departing from the lanes. Some sort of central line marker that can be used to detect if the robot has moved away from the center. I'm still new to OpenCV and python and the part where I'm stuck at is converting the logic of detecting the lane departure of the robot.

My understanding is that averaging the lines on the lanes into two lines (left and right lanes) and then working with their slopes should give some result. However, I've not been able to transform this into code. I'd appreciate any suggestions on ways to detect lane departure of the robot. Thanks!! :)

Answer 1

One of the best ways is to use a combination of hough-transform (what you have achieved is nice) and inverse perspective transform (as in http://www.vision.caltech.edu/malaa/publications/aly08realtime.pdf). This is because, inverse perspective transform fails at 'very near' and 'very far' distances and hough transform can be compensated for that.

Now, once we get parallel lines, we can use that information for localisation; if one of the two edges is missing, then, it means that the robot is not on lane most probably; We can figure out mid-line (average of two edges after inverse perspective transform and filtered by hough-line fusion) and use an evaluation function to figure out how distant the robot is from the centre of the lane.

This solution definitely fails in cases of intersection and parallel lanes (one lane next to another and both lanes are detected simultaneously). In such cases, some sort of mapping or path planning is mandatory (with costs associated for change of lane and wrong routes at intersections). Thus, this solution only applies locally.

For some more learning, auto-encoders can be used for steering angle estimation (as in https://arxiv.org/pdf/1608.01230.pdf ; But, too much blurring can mean simple averaging and very precise auto-encoding reproduction means over-fitting; Dataset can be found here https://github.com/commaai/research).