Machine Learning by Human Hand

Programming robots for industrial activities has been a lengthy process to date. After writing the code, testing is next, followed by the rewriting, until the machine performs a process reliably. These learning phases are too long for the production technologies of the future, which depend on rapid adaptability. "In future, making a robot ready for use must not take longer than training a new employee," says Edgar Welte. In order to achieve this goal, the team wants to teach the robots to learn in the same way as a human learns. In the same way that trainees learn from their masters, the team’s robotic gripper system is intended to learn from an operator how to use new tools, handle different materials, perform entire work steps, and react to changes in production processes. "We use autonomous imitation learning and deep reinforcement learning for our system. The robot learns the skills intuitively and immediately through the interaction with a human," Rania Rayyes explains. This approach reduces retooling times in addition to reducing the original programming process.

Human learning is an exchange of information. The participants talk to each other, interpret visual information, and ask questions. In the HaptXDeep Future Laboratory, the researchers communicate with the robot via data. Edgar Welte uses his movements in the HaptX glove to explain to the six-axis Cobot how to rotate and pack tools or workpieces. Thanks to the 20 servo motors, the Shadow Dextorous hand is able to simulate human gripping movements almost exactly - better than any other gripper arm on the market. The pressure sensors in mechanical fingers give the operator direct feedback. Through small air cushions in the HaptX glove, Edgar Welte can feel it if the weight increases or the tool is incorrectly positioned in the hand. Then he immediately corrects his grip and the robot-gripper does the same. The researcher corrects the robot's error in real time. This, too, is an important function for the machine to be able to learn independently later, based on Artificial Intelligence. "The immediate corrections allow us to quickly expand the capabilities of our system and save us months of work on reprogramming," explains Rania Rayyes.

The HaptXDeep Future Laboratory has a dual function for the project. It is the hardware for the research and development of software-based technologies and innovation, and at the same time it is the demonstrator for their functionality. "It’s a unique opportunity for me to be able to use such a high-quality laboratory for my research," says Edgar Welte. With his doctoral degree studies on the topic of "Interactive Imitation Learning for Dexterous Manipulation", he will make a vital contribution to the development of the overall system. In addition, HaptXDeep will also be used for projects on sensor technology, alternative teleoperation methods such as gesture recognition via motion tracking, and controlling the various fingers.

This will also include collaborations with companies such as Bosch AI or other university institutes. "HaptXDeep is intended to open as many doors as possible in research," says Rania Rayyes. One of them leads to the group of Junior Professor Andrey Morozov at the IAS at the University of Stuttgart. Within the framework of cooperation projects, the researchers in Stuttgart will develop methods to ensure the safety and reliability of the adaptable robot. Fault injection via digital twins will be used as a test procedure for the system, while deep learning systems will help the robot to detect anomalies and predict errors. The robot is intended to be able to correct errors autonomously at some point on the basis of what it has learned. This part of the software is being developed in Stuttgart and tested in the Reality Lab of the IFL at KIT. HaptXDeep is a state-of-the-art research infrastructure as well as an interface through which ICM brings together the best researchers in the country to jointly find groundbreaking solutions for human robot learning.

Editorial and public relations, InnovationCampus Future Mobility

E-Mail: medien(at)icm-bw.de