News

Machine Learning by Human Hand

- News

Two institutes, one at the Karlsruhe Institute of Technology and one at the University of Stuttgart, are jointly developing an adaptable robot that has a gripper system and learns human abilities through imitation. For this purpose, the scientists have set up a research laboratory that is unique in Germany: the HaptXDeep Future Laboratory, financed by funds of the InnovationCampus Future Mobility (ICM).

Edgar Welte is grasping the air with his fingers and brings the robot one movement closer to being human. That is what it should be like, at least in the near future. With the HaptX virtual reality glove, the researcher still controls the robot and its humanoid Shadow Dexterous hand remotely, which is grasping a tool next door. However, the doctoral student and Junior Professor Rania Rayyes set up the ICM HaptXDeep Future Laboratory at the Institute for Material Handling and Logistics (IFL) at KIT in order to develop a robotic gripper system that learns human activities through imitation. Although the first laboratory in Germany with the complete system of the company Shadow Robot is located in Karlsruhe, the Institute of Industrial Automation and Software Engineering (IAS) at the University of Stuttgart is also involved in the research. The two institutes will jointly develop an autonomously learning robot that has a gripper system and is able to react quickly and flexibly, but also reliably and safely, to changing requirements, product designs, and materials.

© Amadeus Bramsiepe, KIT

Programming robots for industrial activities has been a lengthy process to date. After writing the code, testing is next, followed by the rewriting, until the machine performs a process reliably. These learning phases are too long for the production technologies of the future, which depend on rapid adaptability. "In future, making a robot ready for use must not take longer than training a new employee," says Edgar Welte. In order to achieve this goal, the team wants to teach the robots to learn in the same way as a human learns. In the same way that trainees learn from their masters, the team’s robotic gripper system is intended to learn from an operator how to use new tools, handle different materials, perform entire work steps, and react to changes in production processes. "We use autonomous imitation learning and deep reinforcement learning for our system. The robot learns the skills intuitively and immediately through the interaction with a human," Rania Rayyes explains. This approach reduces retooling times in addition to reducing the original programming process.

Human learning is an exchange of information. The participants talk to each other, interpret visual information, and ask questions. In the HaptXDeep Future Laboratory, the researchers communicate with the robot via data. Edgar Welte uses his movements in the HaptX glove to explain to the six-axis Cobot how to rotate and pack tools or workpieces. Thanks to the 20 servo motors, the Shadow Dextorous hand is able to simulate human gripping movements almost exactly - better than any other gripper arm on the market. The pressure sensors in mechanical fingers give the operator direct feedback. Through small air cushions in the HaptX glove, Edgar Welte can feel it if the weight increases or the tool is incorrectly positioned in the hand. Then he immediately corrects his grip and the robot-gripper does the same. The researcher corrects the robot's error in real time. This, too, is an important function for the machine to be able to learn independently later, based on Artificial Intelligence. "The immediate corrections allow us to quickly expand the capabilities of our system and save us months of work on reprogramming," explains Rania Rayyes.

The HaptXDeep Future Laboratory has a dual function for the project. It is the hardware for the research and development of software-based technologies and innovation, and at the same time it is the demonstrator for their functionality. "It’s a unique opportunity for me to be able to use such a high-quality laboratory for my research," says Edgar Welte. With his doctoral degree studies on the topic of "Interactive Imitation Learning for Dexterous Manipulation", he will make a vital contribution to the development of the overall system. In addition, HaptXDeep will also be used for projects on sensor technology, alternative teleoperation methods such as gesture recognition via motion tracking, and controlling the various fingers.

This will also include collaborations with companies such as Bosch AI or other university institutes. "HaptXDeep is intended to open as many doors as possible in research," says Rania Rayyes. One of them leads to the group of Junior Professor Andrey Morozov at the IAS at the University of Stuttgart. Within the framework of cooperation projects, the researchers in Stuttgart will develop methods to ensure the safety and reliability of the adaptable robot. Fault injection via digital twins will be used as a test procedure for the system, while deep learning systems will help the robot to detect anomalies and predict errors. The robot is intended to be able to correct errors autonomously at some point on the basis of what it has learned. This part of the software is being developed in Stuttgart and tested in the Reality Lab of the IFL at KIT. HaptXDeep is a state-of-the-art research infrastructure as well as an interface through which ICM brings together the best researchers in the country to jointly find groundbreaking solutions for human robot learning.

Contact

 

Benjamin Büchner

Editorial and public relations, InnovationCampus Future Mobility

E-Mail: medien(at)icm-bw.de

Back