Teleoperation of a humanoid robot using Oculus Rift and Leap Motion

Abstract

This thesis presents a system to operate a humanoid robot from a distance. Teleoperated robots are used in environments inaccessible to humans for tasks which cannot be performed autonomously. Because of the many degrees of freedom of humanoid robots, manual teleoperation methods with keyboards, joysticks and screens are cumbersome to use. The introduced solution is a first-person teleoperation application where the robot imitates the movements of the user’s upper body. The focus is on providing an intuitive user experience with commercially available consumer-grade electronics: the Oculus Rift virtual- reality headset with the Leap Motion 3D hand tracking sensor mounted. The user receives visual feedback from the robot’s cameras inside the Oculus Rift which tracks the user’s head pose. The robot¿s head pose and therefore the camera direction is controlled intuitively by the user turning his head. We introduce a method for a representation of the video feed in a virtual 3D space to avoid cyber-sickness. The user’s arm movements are tracked by the Leap Motion sensor. We derive the necessary robot joint angles for natural imitation of the user¿s arm posture from the tracked joint positions and direction vectors. The system is able to imitate human motions with high precision and low latency. It was implemented for the humanoid robots Romeo and Pepper manufactured by SoftBank Robotics. Over 100 users tested the system with each robot. Each of them was able to control the robots’ upper-body within seconds without any training.

Publication
Technische Universität Wien Diplomarbeiten