1
$\begingroup$

I would like to make a robotic system which takes as input a video feed, runs some GPU-based image recognition on the video, and outputs commands to a set of motors. The goal is to have the motors react to the video with as little latency as possible, hopefully of the order of 10s of ms. Currently I have a GTX 770m on a laptop running Ubuntu 14.04, which is connected to the camera and doing the heavy image processing. This takes frames at 30Hz and will output motor commands at the same frequency.

After a few days of looking around on the web for how to design such a system, I'm still at a loose end whether (a) it is even feasible (b) if so, what the best approach is to interface the laptop with the motors? The image processing must run on Linux, so there is no leeway to change that part of things.

$\endgroup$
0
$\begingroup$

So I mainly use a ZED camera with a Nvidia K1, using the provided plugins for OpenCV GPU compute and I get a processing time of around 16-20ms for low res images (e.g. 640x480, or 1024x768) and use this to control a flight controller, using a uart interface, I assume your motors take pwm input, as for controlling the motors I would suggest somthing like the polou maestro to control them. Can you give more information on your application? why do you have these image processing requirements?

$\endgroup$
  • $\begingroup$ Thanks very much! The application is to try to control a ping pong paddle to keep a ball bouncing. I was planning to use some deep learning for image processing (hence the GPU), to see whether I can get better results than hand-engineering a ball tracker etc. It's more of an academic exercise :) Could I ask do you think a uart is necessary in my application? It seems like I can directly control the polou maestro (and therefore motors) from the laptop via USB? $\endgroup$ – LPlates May 28 '15 at 12:59
  • $\begingroup$ Yep you can control directly via usb, should have the reaction speed your looking for, I have used them for quadrocopters which are very latency sensitive without issue $\endgroup$ – Mark Omo May 28 '15 at 13:00
  • $\begingroup$ Sounds great - will give it a go. Thanks again. $\endgroup$ – LPlates May 28 '15 at 13:26
0
$\begingroup$

Did you have a look into Orocos http://www.orocos.org/ ? Compared with ROS it has the huge advantage of real time capability. That would be my choice to implement complicated real time controllers.

In my work (http://www.gpu-voxels.org) I am also using a GPU for 3D collision avoidance calculations on our robots, but I am using our own MCA2 http://www.mca2.org framework for the real time stuff. Our actuators are all controlled via CANOpen interfaces.

30 Hz should be no problem at all! We run some motor controllers with 100 Hz and more. Andy my GPU (NVidia TITAN) processes 3D pointclouds of multiple Kinect cameras at 30Hz. So 2D image processing should be possible, even on a Laptop GPU.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.