I am making a Ping pong-playing robot using two cameras to trace the ball (3D vision),what is the minimum Frame rate of the cameras?

Question

I am making a Ping pong-playing robot, like the one in this video, and i am intending to use two cameras to trace the ball in 3D space. Supposing that the robot is playing against a beginner player, the ball takes about 0.7 second to travel from one side of the table to the other. but during this time the robot should process number of frames and predict the rest of the ball track, and the robotic arm should move to the required position.

I read some papers talking about the same project, but I found a big difference from one to another(some paper used 30 FPS, while other one used 120 FPS). I didn't order the cameras yet, and I can't try more than once because it is a graduating project, so the time and the budget are limited.

So is there any way to predict the minimum Frame rate of the cameras?

I also heard that some projects are using Kinect instead of using two cameras or 3D vision,

Is the Kinect fast and precise enough for my project?

Answer 1

The movement of the ping pong ball is going to be ballistic, so you really only need to know its 3d position in 3 different locations in order to fully constrain its motion. In reality, you will probably want more. This excellent paper An Application of human-robot interaction: Development of a ping-pong playing robotic arm uses around 8-10 locations in order to fully constrain the ballistic motion, including the bounce off the table. It also has a review of past systems and approaches.

You really need to think not about the frame rate of the camera but the processing rate of the system.If you say you need 6 3d locations between both sides of the table then having more than:

$$ \frac{6 frames}{.7 sec} = 8.5 fps $$ does not really help you, but it imposes another limitation, you need to be able to capture, transfer and process images in less than: $$ \frac{1 sec}{8.5} = 117ms $$

This usually ends up being the limitation for these kinds of systems.

Additional food for thought, syncing the camera's shutter is usually pretty important or at least sampling the images close to one another. For example with two 30FPS cameras the maximum time between two frames acquired by the computer at the "same" time is 33ms but in the time the ball will have traveled $$ \left(\frac{2.7m}{0.7 sec} * 33ms \right) = 12.8 cm $$

Depending on how your analyzing the data this might be significant (hint it probably is) This is why some people might be using 120fps cameras reducing the max error to around 3.2cm or you might use cameras that have a global shutter clock/sync to make sure that every image captured by all your cameras are taken at the same instant. The Firefly MV camera series is an inexpensive example of cameras that can have their shutters synchronized.

TL;DR

The Kinect is probably fast enough for your purpose, it's certainly cheaper than using Point Grey cameras, so I would give it a shot, I think moving a robot arm fast and accurate enough will be a bigger challenge.

Answer 2

• First you need to calculate the maximum processing period time you can accept, preferably use (motor response time)/5 or less to be sure that the motors move smoothly ..
• also you need to calculate the time you need to process the image because the processor that you use has a processing limit that you need to consider..

so the frame rate you need should be fast enough to achieve the first condition, but it would be waste to make faster than what your processor can handle.
but in general for graduating project 30 FPS is probably enough.. it will provide a good information to predict the trajectory of the ball.