1
$\begingroup$

My team has been working on a wearable glove to capture data about hand movements, and use it as a human-computer interface for a variety of applications. One of the major applications is the translation of sign language, shown here: https://www.youtube.com/watch?v=7kXrZtdo39k

Right now we can only translate letters and numbers, because the signs for them require the person to hold their hand still in one position ('stationary' signs). I want to be able to translate words as well, which are non-stationary signs. Also the position of the hands really matters when signing words, for example it matters whether the hand is in front of the forehead, eyes, mouth, chest, cheeks, etc.

For this we need a portable and highly accurate position sensor. We have tried getting position from a 9-DOF IMU (accelerometer, gyroscope, magnetometer) but as you might guess, there were many problems with double integration of the noise and accelerometer bias.

So is there a device that can provide accurate position information? It should be portable and wearable (for example worn in the chest pocket, headband/cap, etc...be creative!).

EDIT (more details):

I'm going to emphasize certain aspects of this design that weren't clear before, based on people's comments:

  1. My current problem of position detection is due to errors in double integration of the accelerometer data. So hopefully the solution has some incredibly powerful kalman filter (I think this is unlikely) or uses some other portable device instead of an accelerometer.
  2. I do not need absolute position of the hand in space/on earth. I only need the hand position relative to some stable point on the body, such as the chest or belly. So maybe there can be a device on the hand that can measure position relative to a wearable device on the body. I don't know if such technology exists; I guess it'd use either magnets, ultrasound, bend sensors, or EM waves of some sort. Be creative :)
Improve this question
$\endgroup$
6
  • $\begingroup$ "highly accurate position sensor." How accurate is that exactly? There's roughly 30cm between face and chest. You don't need anything "accurate" for that. Could you provide more information on what positions have to be identified? $\endgroup$
    – Bending Unit 22
    Aug 7, 2016 at 22:18
  • $\begingroup$ I say 'accurate' because I need the position information to be correct without accumulating errors for about 20 minutes to 1 hour. And also it needs to be precise because we need to differentiate between the hand being near the nose, eyes, mouth, etc. I wish I could give you a numerical specification but to be honest I will take the best of what's available. My priority is accuracy rather than precision, i.e. I don't need anything more precise than 2-5cm but it should not report wrong position after continuous use for 20-60 minutes. $\endgroup$
    – Bhavesh
    Aug 8, 2016 at 1:04
  • 2
    $\begingroup$ It sounds like you're more concerned about drift caused by accumulation of errors over time, which is a valid concern. This can best be handled by a kaman filter, rather than a more accurate sensor. $\endgroup$
    – Paul
    Aug 8, 2016 at 3:49
  • $\begingroup$ hes worried about observability, which is the problem he ran into with the double integration $\endgroup$
    – holmeski
    Aug 8, 2016 at 12:56
  • $\begingroup$ @Paul Is it possible to have a kalman filter for this application? There are a lot of jerky hand movements in all sorts of directions so will the filter be able to provide an accurate position for 20-60 mins? $\endgroup$
    – Bhavesh
    Aug 10, 2016 at 2:42

2 Answers 2

Reset to default
1
$\begingroup$

The problems of outdoor-tracking are described in http://www.tinmith.net/papers/piekarski-ismar-2006.pdf page 2. As a solution was named the WearTrack System which worked by ultrasonic technology. The system tracks the hand relative to the head of the user. Other hardware solutions are presented here https://www.evl.uic.edu/jelias1/vr-shapetape.pdf (ShapeTape Tracker) and here https://www.xsens.com/images/stories/PDF/Combining%20Motion%20Sensors%20and%20Ultrasonic%20Hands%20Tracking.pdf (ultrasonic tracker with gesture recognition).

Improve this answer
$\endgroup$
2
  • $\begingroup$ The indoor cyberglove mentioned is not portable at all, although quite a capable and interesting high tech device. I am looking at the papers under 'outdoor tracking' which are very cool solutions, especially the WearTrack system! $\endgroup$
    – Bhavesh
    Aug 10, 2016 at 1:02
  • $\begingroup$ I had a chance to read all of these and the papers in your Outdoor tracking section are quite helpful! I am willing to mark it as the correct answer except there is a lot of information that doesn't answer the question (namely, the indoor cyberglove section). Also it is important to write a bit (at least a sentence) about the "shapetape" paper and the xsens paper. $\endgroup$
    – Bhavesh
    Aug 12, 2016 at 12:28
0
$\begingroup$

Manuel Rodriguez's idea for the strip-flex potentiometers is about as good are you are going to get. Accurate 3D mapping is not something easily accomplished, let alone accomplished in a portable package. The only other solution I can think of is something like the Leap Motion - but that is far from portable.

Watching their video, they seem to be using the same thing for not just the fingers, but the wrist and elbow joints as well. I can't tell, but it looks like they might also be running OpenCV (or similar) alongside this as well.

The trouble is that it sounds like you aren't even sure exactly what the intended application of this technology is. Not trying to be patronizing but what is the goal here? Why do you need an ASL-computer interface, and why does it need to be portable? Someone who needs to ASL for interpersonal communication should have no trouble interfacing with the average computer or smart phone - so I am assuming this is to allow them to communicate with people who do not know ASL (via gesture-to-computer-generated-speech)? If this is the case, then perhaps the solution you have picked is an overly complicated one to your problem.

I would suggest you revisit the drawing board all-together, and come up with a different approach. Perhaps a computer-vision application that can interpret signs, and run on a smart phone? Probably quicker, easier, and more accurate than trying to tackle a hardware problem that I have yet to see outside of a PhD thesis project in one of the world's top robotics research labs (and the one video on Youtube you linked to). Plus, such a solution could probably be later integrated into one of the existing 'universal translators' that Microsoft and Google are working on.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I should've mentioned in my post, the youtube video I linked to was a video of my team and I demonstrating our first prototype of this glove for our undergraduate final project. There is also a video that became popular in May showing University of Washington students who'd invented a similar device, again with no computer-vision or cameras. I am aiming for it to be portable so that non-verbal people can use this anywhere outside and communicate with anyone, without having to face a screen. Also ASL requires using both hands so they would not be able to hold a phone while communicating. $\endgroup$
    – Bhavesh
    Aug 10, 2016 at 13:43
  • $\begingroup$ Alright, that is pretty impressive. But I don't think that alleviates my initial concerns that you might be inventing an overly complicated solution for this particular problem. Trouble is that a single human arm has something like 27~ DOFs, and ASL uses both arms. Accuracy will not be easy, regardless of what sensors you use, if you use an Electromechanical system. Never mind getting it fast enough for most users to want to use it. $\endgroup$
    – MechanicalMan
    Aug 10, 2016 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.