Planar Robot Boundary Detection

Question

I am tasked with building a robot that needs to localise and navigate itself on a rectangular surface (like a large desk). The rectangle's shorter sides will be marked with a red tape (or with two different colours if need be), while the longer edges are marked with a black tape. My robot is fitted with a forward facing camera, orientated downward slightly, as well as 6 sonar sensors around it's perimeter to detect any of the edges of the rectangle. There is also an IMU and encoders on each wheel (this is a two wheeled robot, with the third point of contact being a trolley wheel). The size of the rectangle is known and thus a map of it is available.

My questions are thus these:

1. Using the camera( I have both a wide angle cam or a standard camera at my disposal) how would I be able to find useful information from the edges of the rectangle. Obviously not all the rectangle edges will be in frame of the camera simultaneously, but what ros packages/methods could I use to gather information from them using the different colours of tape.

2. If/once I have information about the edges, are there any navigation tools that would allow me to do a sweep over the whole mirror (such as a lawnmower over a garden) to navigate the whole surface area.

3. What is the simplest way to get the sonar's to prevent the robot driving off the edge, given that they are downward facing?

The biggest challenge here seems to be that there is no real absolute reference, as the nature of the project requires the robot to localize itself only from data on the rectangle.

Answer 1

1. Since the lines are (usually) going to be perpendicular to the normal vector of your image, you can sample columns and use a Butterworth filter to extract the "peak" frequencies (must be configured towards the desired line coloration). Converting to a gray scale first is typically more effective. This will give you the location of the line in your image. To find the exact color, just search the image at that specific location and check the RGB values (using an admissible range).

For dealing with corners, you can apply the filter to both a column and row and check both. If you are using python, you can see the SciPy Butterworth Filter.

2. Assuming you have a sufficiently accurate controller, you can solve this issue with way-point navigation and short term dead reckoning (which is where IMUs are most effective).

3. To answer this part (accurately), we need more information about the physical configuration of your sonar sensors w.r.t. the rest of your system.