image: Artificial tactile perception allows the brain-computer interface user to transfer objects with a robotic arm at twice the speed of doing it without the feedback. view more
Credit: UPMC/PITT Health Sciences
PITTSBURGH, May 20, 2021 - Most able-bodied people take their ability to perform simple daily tasks for granted--when they reach for a warm mug of coffee, they can feel its weight and temperature and adjust their grip accordingly so that no liquid is spilled. People with full sensory and motor control of their arms and hands can feel that they've made contact with an object the instant they touch or grasp it, allowing them to start moving or lifting it with confidence.
But those tasks become much more difficult when a person operates a prosthetic arm, let alone a mind-controlled one.
In a paper published today in Science, a team of bioengineers from the University of Pittsburgh Rehab Neural Engineering Labs describe how adding brain stimulation that evokes tactile sensations makes it easier for the operator to manipulate a brain-controlled robotic arm. In the experiment, supplementing vision with artificial tactile perception cut the time spent grasping and transferring objects in half, from a median time of 20.9 to 10.2 seconds.
"In a sense, this is what we hoped would happen--but perhaps not to the degree that we observed," said co-senior author Jennifer Collinger, Ph.D., associate professor in the Pitt Department of Physical Medicine and Rehabilitation. "Sensory feedback from limbs and hands is hugely important for doing normal things in our daily lives, and when that feedback is lacking, people's performance is impaired."
Study participant Nathan Copeland, whose progress was described in the paper, is the first person in the world who was implanted with tiny electrode arrays not just in his brain's motor cortex but in his somatosensory cortex as well--a region of the brain that processes sensory information from the body. Arrays allow him to not only control the robotic arm with his mind, but also to receive tactile sensory feedback, which is similar to how neural circuits operate when a person's spinal cord is intact.
"I was already extremely familiar with both the sensations generated by stimulation and performing the task without stimulation. Even though the sensation isn't 'natural'--it feels like pressure and gentle tingle--that never bothered me," said Copeland. "There wasn't really any point where I felt like stimulation was something I had to get used to. Doing the task while receiving the stimulation just went together like PB&J."
After a car crash that left him with limited use of his arms, Copeland enrolled in a clinical trial testing the sensorimotor microelectrode brain-computer interface (BCI) and was implanted with four microelectrode arrays developed by Blackrock Microsystems (also commonly referred to as Utah arrays).
This paper is a step forward from an earlier study that described for the first time how stimulating sensory regions of the brain using tiny electrical pulses can evoke sensation in distinct regions of a person's hand, even though they lost feeling in their limbs due to spinal cord injury. In this new study, the researchers combined reading the information from the brain to control the movement of the robotic arm with writing information back in to provide sensory feedback.
In a series of tests, where the BCI operator was asked to pick up and transfer various objects from a table to a raised platform, providing tactile feedback through electrical stimulation allowed the participant to complete tasks twice as fast compared to tests without stimulation.
In the new paper, the researchers wanted to test the effect of sensory feedback in conditions that would resemble the real world as closely as possible.
"We didn't want to constrain the task by removing the visual component of perception," said co-senior author Robert Gaunt, Ph.D., associate professor in the Pitt Department of Physical Medicine and Rehabilitation. "When even limited and imperfect sensation is restored, the person's performance improved in a pretty significant way. We still have a long way to go in terms of making the sensations more realistic and bringing this technology to people's homes, but the closer we can get to recreating the normal inputs to the brain, the better off we will be."
###
Additional authors of this study include Sharlene Flesher, Ph.D., Jeffrey Weiss, M.S., Christopher Hughes, M.S., Angelica Herrera, B.S., and Michael Boninger, M.D., all of Pitt; John Downey, Ph.D., of the University of Chicago; and Elizabeth Tyler-Kabara, M.D., of the University of Texas at Austin.
This work was supported by the Defense Advanced Research Projects Agency (DARPA) and Space and Naval Warfare Systems Center Pacific (SSC Pacific) under Contract No. N66001-16-C-4051 and the Revolutionizing Prosthetics program (Contract No. N66001-10-C-4056).
To read this release online or share it, visit https://www.upmc.com/media/news/052021-bci-stim-touch-aaas [when embargo lifts].
About the University of Pittsburgh School of Medicine
As one of the nation's leading academic centers for biomedical research, the University of Pittsburgh School of Medicine integrates advanced technology with basic science across a broad range of disciplines in a continuous quest to harness the power of new knowledge and improve the human condition. Driven mainly by the School of Medicine and its affiliates, Pitt has ranked among the top 10 recipients of funding from the National Institutes of Health since 1998. In rankings recently released by the National Science Foundation, Pitt ranked fifth among all American universities in total federal science and engineering research and development support.
Likewise, the School of Medicine is equally committed to advancing the quality and strength of its medical and graduate education programs, for which it is recognized as an innovative leader, and to training highly skilled, compassionate clinicians and creative scientists well-equipped to engage in world-class research. The School of Medicine is the academic partner of UPMC, which has collaborated with the University to raise the standard of medical excellence in Pittsburgh and to position health care as a driving force behind the region's economy. For more information about the School of Medicine, see http://www.medschool.pitt.edu.
Contact: Anastasia Gorelova
Mobile: 412-491-9411
E-mail: GorelovaA@upmc.edu
Contact: Sheila Davis
Mobile: 412-313-6070
E-mail: DavisSN2@upmc.edu
Journal
Science