ROS: Human to Robot Mapping (Baxter)

Question

I'm having some difficulties understanding the concept of teleoperation in ROS so hoping someone can clear some things up.

I am trying to control a Baxter robot (in simulation) using a HTC Vive device. I have a node (publisher) which successfully extracts PoseStamped data (containing pose data in reference to the lighthouse base stations) from the controllers and publishes this on separate topics for right and left controllers.

So now I wish to create the subscribers which receive the pose data from controllers and converts it to a pose for the robot. What I'm confused about is the mapping... after reading documentation regarding Baxter and robotics transformation, I don't really understand how to map human poses to Baxter. I know I need to use IK services which essentially calculate the co-ordinates required to achieve a pose (given the desired location of the end effector). But it isn't as simple as just plugging in the PoseStamped data from the node publishing controller data to the ik_service right?

Like a human and robot anatomy is quite different so I'm not sure if I'm missing a vital step in this.

Seeing other people's example codes of trying to do the same thing, I see that some people have created a 'base'/'human' pose which hard codes co-ordinates for the limbs to mimic a human. Is this essentially what I need? Any insight is very much appreciated!

Answer 1

I answered your question on stack overflow: https://stackoverflow.com/questions/51385737/ros-human-robot-mapping-baxter

You might find my former student's work on motion mapping using a kinect sensor with a pr2 informative. It shows two methods:

1. Direct joint angle mapping (eg if the human has the arm in a right angle then the robot should also have the arm in a right angle).

2. An IK method that controls the robot's end effector based on the human's hand position.

I know I need to use IK services which essentially calculate the co-ordinates required to achieve a pose (given the desired location of the end effector). But it isn't as simple as just plugging in the PoseStamped data from the node publishing controller data to the ik_service right?

Yes, indeed, this is a fairly involved process! In both cases, we took advantage of the kinects api to access the human's joint angle values and the position of the hand. You can read about how Microsoft research implemented the human skeleton tracking algorithm here:

https://www.microsoft.com/en-us/research/publication/real-time-human-pose-recognition-in-parts-from-a-single-depth-image/?from=http%3A%2F%2Fresearch.microsoft.com%2Fapps%2Fpubs%2F%3Fid%3D145347

I am not familiar with the Vive device. You should see if it offers a similar api for accessing skeleton tracking information since reverse engineering Microsoft's algorithm will be challenging.