Why was the Shadow robot hand been utilized for teleoperation?

Question

With a deep neural network it's possible to convert a camera signal into the movement trajectory of a dexterous robotics hand.[1] In contrast to previous attempts in teleoperation, which are working with a dataglove or a joystick, the visual shape of the hand controls the robot. The human operator uses it's own hand to make a hand sign and the Shadow hand is doing the same action.

In this setup, a human operator is in the loop, which means, it has nothing to do with Artificial Intelligence. A real AI system would control the robot with software, a motion planner and can grasp objects by it's own. If the sensor signals are taken from a human, the system is only semi-autonomous. Does this make sense if the aim is to program autonomous robots? Wouldn't it be better to program an AI which controls the hand autonomously?

[1] Li, Shuang, et al. "Vision-based teleoperation of shadow dexterous hand using end-to-end deep neural network." arXiv preprint arXiv:1809.06268 (2018).

Answer 1

Teleoperation is about letting the human do what humans do best, and letting the robot do what automation does best. There are some situations in which it would take too long, or cost too much, to construct an effective AI system to do certain tasks. Check out, for example, Betczy’s work at JPL, Sheridan’s at MIT, and Hamel’s at Oak Ridge. The idea is to not need to program the entire human intelligence about the task, but instead allow the operator to respond to dynamic task scenarios as if they were present at the task, while taking advantage of the increased strength, stability, and the like from the robot.

As AI continues to progress, the line marking the efficiency of implementing telerobots vs complete automation will continue to move toward increasing amounts of automation. But it will be very challenging to efficiently build an AI engine that can accommodate the unexpected things that a human can handle, in a reasonable amount of time, when the tasks are quite variable and in dynamic environments (think wartime, or catastrophic outside environments, or highly specialized surgeries).