I am working on a robotics project for fun and can't really wrap my head around a solution to my perception problem.
Setup: My robot will have a stereo vision setup and will have to detect certain objects and align itself to those objects in a certain pose. The robot will know what the width and height of those objects are. The robot will be using a tx1 for computation, so implementation needs to be pretty fast. Also, the environment's lightning will change a lot so using color for detection isn't a great option.
My plan: To use convolutional neural networks to detect those objects of interest. I have been able to program a network to detect those objects in 2D however, I am stuck on how to detect the pose of the objects in 3D. My idea has been to detect the object with the neural network and once I have that region of interest, get the point cloud. Then fit a 3D model using ICP. Once the 3D model is fit I can get the pose of the object.
I have also seen people using 3D correspondence grouping for this but would that work on a non dense point cloud that stereo vision generates?
I am pretty novice in this area and would love to get advice on from some more advanced robotics practitioners.
Thanks for you time!