USC team develops new sensory robot hand advancements

Growing up, we learn to push just hard enough to move a box and to avoid touching a hot pan with our bare hands. Now, a robot hand has been developed that also has these instincts.

The MOTIF Hand, developed by a student team in collaboration with Daniel Seita, a USC Viterbi assistant professor of computer science, is built on the idea of being multimodal — that is, having several sensory abilities. The most prominent of these abilities relate to temperature and force, with built-in sensors for depth, force and temperature allowing the hand to sense and react to these factors.

These capabilities create potential not only for better research involving robotic hands, but they also allow these hands to last longer by avoiding temperature-related damage. Force-related capabilities could also have a surprisingly practical real-world use.

“In factories and other domains, a robot would have to push to get objects into their targets, and that requires measuring some amount of force,” Seita said. “That type of force sensor can help in those cases, just to check that the robot is exerting the right amount of force.

“We haven’t seen people build this type of hand before,” he added.

Hot stuff

The MOTIF Hand builds on the foundation of the LEAP Hand, which was built by a research team at Carnegie Mellon in 2023. MOTIF’s key advancement is the addition of human-like sensory capabilities. This MOTIF Hand, which contains far more accurate human-like features and abilities, could have myriad applications, including in factory work and even cooking or welding, Seita said.

The robot’s ability to sense temperature comes from a thermal camera built into the palm of the hand. Seita and his team of USC Viterbi graduate students aimed to create a hand that would simulate a human understanding of temperature.

“If we’re cooking, we have a pot that’s very hot. We might put our hand near it to check if it’s safe to touch before we actually touch it, to avoid burns and damage,” Seita said. “We wanted that same intuition conveyed into a robot system.”

It’s an intuitive system that requires the hand to be close to the material whose temperature it’s detecting said Hanyang Zhou, a co-author of the research paper, “The MOTIF Hand: A Robotic Hand for Multimodal Observations with Thermal, Inertial, and Force Sensors,” who recently graduated from the Viterbi School with a master’s in computer science.

“We were thinking, Is it possible in some certain way to get a signal but not touch anything? So, we put an infrared-based camera right in the palm,” he added.

In other words, the MOTIF Hand can detect temperature through this thermal camera without even touching an object — just placing the hand close enough for the camera to examine it does the job.

“You have to feel it”

The work done by Seita, Zhou and their team was designed to make the process of testing temperature and force feel more natural — in other words, true to human experiences with these things. For example, force is something that humans can’t see, just feel. The MOTIF Hand is designed around the same sensations we use to understand force-related properties, such as an object’s weight, allowing for more life-like robotic reactions to force.

“We as humans cannot distinguish [force] as a vision you have to feel it. But how is that possible for a robot hand?” Zhou asked. “If I don’t know whether a water bottle is full of water, I just flick it. I’ll shake it, right?”

The IMU sensors built into the MOTIF Hand bring this simple test to robotics. The hand, like our own, merely needs to flick or shake an object to determine its weight.  

The MOTIF Hand was based on Carnegie Mellon’s LEAP Hand, which was open source. To further advance this sensory technology, Seita and his team have promised to make the MOTIF Hand open-source as well.

“Open-sourcing research advancement is really important to advance the community,” Seita said. “The more people that use our hand, the better it is for research.”

Zhou described the MOTIF Hand’s sensory advancements as a “platform” that he hopes the entire robotics community will build on for the future.

“We should make it easy [and] accessible for more and more research teams, as long as they are interested in such a platform,” Zhou said.