I have an object like attached in image.

enter image description here

It is hanging vertical (its position may change but static object) and I want to grasp it using 6DOF Robotic arm. I am Using camera to detect object.

I am trying to estimate its 6D pose w.r.t to camera. The question is do I need to estimate 6D pose of object in order to grasp it? Is that really required, or is my thinking incorrect?

What are the different ways to grasp it using camera?

Can I use just distance between camera and object (after detection), and tell robot to pick at that distance in the middle by gripper? If this is possible, how can I give value of X;Y;Z; ROLL; PITCH; YAW for the pick?

Can anyone guide me to correct path?

  • 1
    $\begingroup$ Welcome to Robotics user22120, but I'm afraid that there are many ways to solve any given design problem, so questions like this are off-topic. We prefer practical, answerable questions based on actual problems that you face, so it's a good idea to narrow down your question to a single concrete problem, including details of what you want to achieve, what you tried, what you saw & what you expected to see. Please take a look at How to Ask & tour for more information on how stack exchange works and edit your question to make it clearer. $\endgroup$
    – Mark Booth
    Apr 5, 2019 at 11:17

1 Answer 1


What you're trying to do is also known as Visual Servo. You could do it only with the position of the object, but it's normally not done this way because without the rotation information, you cannot guarantee Force nor Form Closure, which is what you need if you don't want to drop your object, be it because of external forces, slippery, etc.

You first need to transform the information in relation to the camera to information in relation to the robot. To do this, you usually need the transform from the robot to the camera. This is a simple transformation. If you have the robot pose in relation to world, and the camera pose in relation to world, then:

enter link description here

Where T is the homogeneous transformation, and R, C and O represent the Robot, Camera and Object Frame.

If your World frame is the Robot Frame:

enter link description here


enter link description here

For homogeneous transformation I suggest you take a look at Robotics, Vision and Control Second Edition by Peter Corke.

After that, it becomes a grasping problem.

  • $\begingroup$ thank you for answer. so Object 6D(3r, 3t) pose wrt camera using image processing is must in my object? $\endgroup$
    – user22120
    Apr 4, 2019 at 15:19
  • $\begingroup$ It actually depends on your gripper. But generally, if you want to guarantee that the object won't fall or slipper, you need to guarantee force or form closure, and that normally needs the object 6D. For the object in the picture, I'd say yes, it's a must. $\endgroup$ Apr 4, 2019 at 15:54
  • $\begingroup$ can I use just depth information where I can say camera will on robot hand and always grasping in middle of part, then tell the robot to grasp object at that depth? $\endgroup$
    – user22120
    Apr 5, 2019 at 8:08
  • $\begingroup$ It's possible. The problem is the rotation of the object. If rotated in a certain way, your gripper might not be able to grasp it if you're using a fixed rotation. $\endgroup$ Apr 5, 2019 at 12:01
  • $\begingroup$ it is hanging on fixed location.and will not rotate. $\endgroup$
    – user22120
    Apr 5, 2019 at 12:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.