Abstract
Dexterous robotic hands are necessary for many tasks requiring the capability of in-hand object manipulation or tool use. Teleoperation remains a common approach for controlling dexterous hands, however, it is very difficult with traditional input methods (joystick, keyboards, etc.) or requires substantial investment in hardware setup (e.g. sensorized gloves). We introduce a teleoperation system based on the consumer-grade Leap Motion sensor to control a simulated model of the Shadow Dexterous Hand. Further, we present two continuous control tasks for the open source simulator PyBullet, integrated with the OpenAI Gym interface. We evaluate our system by using trajectories recorded by human experts as a supervised pre-training step before applying reinforcement learning (RL) algorithms. We show that the use of expert demonstrations accelerates the RL training process. The proposed system is available at: https://rgit.acin.tuwien.ac.at/matthias.hirschmanner/shadow_teleop/.