Can I use IMUs to improve the position/posture measurement of fingers in a “data glove”?

Question

I have been using the Cyberglove to control a humanoid robot hand, but found it disappointing as it doesn't measure the posture of the human hand very accurately.

I have been wondering about the possibility of using Inertial Measurement Units (IMUs) mounted on the fingers to track position and measure posture. But I'm not sure how feasible it is.

• Would an IMU return enough data to make tracking reliable in all circumstances?
• Would it be possible to fool the system into incorrectly tracking the fingers?
• Might it be possible to get away with using simple 3-axis accelerometers, or would it need 9-axis (accelerometer, gyro, and magnetometer)?

Answer 1

Simple 3-axis accelerometers will give you an estimate of pitch. However, this will be very noisy, especially when the hand is moving from one pose to another. This is because the direction of acceleration is directly used to estimate the direction of gravity. Thus, this option would only be useful if the hand is not moving.

Gyroscopes allow the direction of gravity to be stabilized. This allows the data to remain useful during movement. Drift/bias is not a problem, so long as you correct for this. This can be done by using a Kalman filter, with a model of gyroscopic drift. The drift is corrected by comparing the derivative of the long term estimate of the direction of gravity compared to the gyroscopic data. Some IMUs correct for drift automatically, for example, the VN-100 from VectorNav.

Magnetometers provides a global estimate of the remaining angular direction (yaw). This simply adds further useful data.

If you only place the IMUs on the fingertips, the data is insufficient to estimate pose. Remember that IMUs only measure attitude (also known as orientation) in space. Can you imagine cases where the fingertip may remain in the same orientation, yet the hand pose is different?

Observe that the index fingertip orientation does not change, however, the position (pose) has changed.

Therefore, you can see that IMU data on the fingertips is not enough to estimate pose. However, additional data can improve the estimate of hand pose, if you fuse the IMU data with your existing data using a Kalman filter.

It is theoretically possible for IMU data alone to be sufficient to estimate pose. However, this would require an IMU for each and every independently movable part of the hand. Since I cannot independently move the top two segments of each finger, we would only need 2 IMUs per finger (one on the fingertip, and one on the segment just after the knuckle, ie. segments 1 and 3 on the finger), plus 2 for the hand, and another for the end of the arm (to get the wrist motion). Therefore, you need a total of 13 IMUs to get all the data required.

I do not know that much about the price of tiny IMUs, however, I would guess that it is not that cheap to get that many IMUs (I don't know your budget). Normally, a single IMU is used for a rigid body. IMUs vary in price (a list at http://damien.douxchamps.net/research/imu/), but say, the VN-100 IMU might be able to give you 0.1 degree accuracy in optimal conditions. If you get a cheaper IMU, the accuracy may not be as good and/or they may lack an integrated Kalman Filter. Getting 13 VN-100 IMUs, costing thousands would probably break the bank (although the VN-100 at 20x20x2mm may not be appropriate if you want an IMU with a smaller form factor).

My suggestion is to consider other data sources which may be obtained more cheaply. For example:

• camera-based systems, after image processing, you can get data about the position of different parts of the hand. Image processing can be aided by using some sort of reflective markers.
• sensors embedded in the fabric of the glove
  • strain gauges
  • SMA wire
  • ...

Answer 2

The short answer is yes, this can work. The long answer is "Yes, but you need to do a lot of sensor fusion".

The technology you're after was conceived about 5 years ago, the academic work is here: https://people.csail.mit.edu/wojciech/MoCap/index.html

They combined the accelerometer data with ultrasonic ranges between the joints to improve the position estimation. In this case, it was for full-body motion capture but doing it for just fingers should be a similar problem.

This project immediately became a company that began making motion capture devices (for 3D movies, video games, etc), but I can't remember the company name. A google search led me to "animazoo", but I don't think that's what it was called before. Maybe it's the same company with a new name. In either case, those are the only relevant resources I know of but they seem to support your idea.

Answer 3

While ronalchn already provided a pretty good answer, I would actually like to take the stance that it should be possible. It would essentially work the same as the full-body tracking devices that XSens is selling. In these applications you perform model fitting, based on the orientation data from your imu. I am not sure how many sensing nodes you would need, but e.g. you could use something like this integrated 6DOF IMU sensor. Using 13 sensors wouldn't be all that implausible anymore, even from a price point of view. I guess you could also do it with just accelerometers, and one full IMU on the wrist, but it would be quite involved. In the end it comes down to the accuracy that you require.