Implementing monocular visual-tactile sensors for robust manipulation

Tactile is an essential information for humans perceiving the world physically. And tactile sensing plays an important role in improving the performance of planning and control for a robotic manipulator, so as to achieve complex robotic manipulations.

 

Although there have been various principles to achieve a tactile sensor, the visual-tactile sensor is becoming a practical means of implementing tactile sensors, since vision is no doubt a low-cost and effective source to provide abundant information. In recent years, owing to the significant advances in visual information processing, the use of vision to build tactile sensors has become one of the concerning areas of the research community.

 

A study recently conducted by a research team from the School of automation, Chongqing University, proposed a framework to build a low-cost, monocular visual-tactile sensor for robotic manipulation tasks. Additionally, they introduced a method for estimating contact locations. The design is low cost and can be processed in a very short time, making it suitable for use as an exploratory study in the laboratory. The paper is published on Sep. 6 in the journal Cyborg and Bionic Systems.

 

Presently, the main visual-tactile sensor principles can be divided into marker-based visual-tactile sensors (MVTSs) and retrographic sensing-based visual-tactile sensors (RSVTSs).

 

The MVTS is very easy to build due to its simple structure and lack of strict light source requirements. Its accuracy depends mainly on the density of the markers and the minimum resolution at which the camera can recognize the markers. Spherical markers are usually adopted for easy detection. The key point is the design of the markers and the acquisition of the spatial location of each marker.

 

The RSVTS, due to retrographic sensing techniques, can make full use of the full camera pixels to obtain high resolution deformation at the elastic layer. Since such devices require an undisturbed lighting system, the camera and the light in such devices are usually enclosed by a chamber environment.

 

Although the RSVTS help to achieve high-precision measurement, the requirement of lighting system, elastic layer, and the customized sensing component still make replication of these designs costly. Therefore, the motivation of this paper is to provide a framework for designing low-cost, easy-to-build marker-based visual-tactile sensor.

 

To ensure that the designed visual-tactile sensor is low cost and easy-to-build, it is necessary to keep the number of components of the sensor as small as possible and to keep the difficulty of preparing each component as little as possible.

 

The elastic layer is what is referred to as “skin” and where the contact between the object and the sensor occurs. Once the object is pressed against the elastic layer, geometric deformation will take place. The camera is the core component used to sense the deformation of the elastic layer via the changes in the captured images. The connectors are used to combine the elastic layer with the camera and then fix them to the manipulator.

 

The elastic layer is where contact occurs. With long-term use, this component will inevitably wear and age. Therefore, materials with characteristics such as wear resistance, ease of preparation, ease of replacement, and low cost should be considered. In this work, the researchers use silicone resin as the elastic layer material.

 

To prepare the elastic layer, we propose the following pipeline: step 1: design the mold. The shape of the mold determines the shape of the elastic layer. Step 2: design the marker. The marker affects the performance of deformation detection. Step 3: manufacture the elastic layer. A series of processes, such as proportioning, mixing, removing air bubbles, and curing, will be taken to obtain the desired elastic layer.

 

After completing the overall design, the team began a case study in which they designed the elastic layer as finger shape, similar in length and size to the distal and middle phalanges of a human finger. With the installation of robot fingers, the entire fingertip can sense tactile information.

 

To keep the shape of the elastic layer as simple as possible The sensor surface is designed as a joint structure of a semi-sphere and an open cylinder. The marks on the elastic layer can be in the form of dome or spherical coordinates.

 

For the preparation of the elastic layer, a mold was made from aluminum. The thickness of the elastic layer is determined by the inner and outer diameters of the mold, in this case, mold A with a diameter of 8.0 and mold B with diameters of 10.0 and 12.0, respectively. And the researchers used a vacuum drying oven and a weighing scale to make the preparation process accurate and repeatable.

 

The authors 3D print a connector to integrate the elastic layer with the camera.

 

The team also proposed a marker-based approach to estimating the location of contacts using ARUCO markers that can simultaneously detect multiple contact areas.

 

Current work has yielded many encouraging results, but there is still room for improvement of visual-tactile sensors in the following aspects: (1) VTSs are generally easy to fabricate compared to non-VTSs, but the lighting system, the elastic layer, and the customized sensing component still make replication of thesedesigns costly. (2) Most of the VTSs adopt a flat contact surface, which makes them less competitive than their non-VTS counterparts.

 

The main advantages of this visual-tactile sensor are as follows: first, it supports the curved surface contact surface and adapt to more complex object shapes. Second, the transparent shell design can make full use of the image information of the visual sensor.

 

Looking forward, the team will focus on further improving the resolution of tactile perception by designing special pattern of markers. They also aim to explore the use of such sensors in applications such as home services.

 

The paper, "Implementing Monocular Visual-Tactile Sensors for Robust Manipulation", was published in the journal Cyborg and Bionic Systems on Sep 6, 2022, at DOI: https://doi.org/10.34133/2022/9797562

 

Reference

Authors: Rui Li, Bohao Peng

Title of original paper: Implementing Monocular Visual-Tactile Sensors for Robust Manipulation

Journal: Cyborg and Bionic Systems

DOI: 10.34133/2022/9797562

 

Affiliations:

1 School of Automation, Chongqing University, Chongqing, China

 

A brief introduction about yourself. (Needed)

About Dr. Rui Li:

Rui Li received the B.Eng degree in automation engineering from the University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 2013 and the Ph.D. degree from the Institute of Automation, Chinese Academy of Science (CASIA), Beijing, China in 2018, respectively. From 2014 to 2018, he served as the Editorial Assistant of Assembly Automation. From 2018 to 2020, he worked as a postdoc researcher with Informatics 6, Technical University of Munich. Currently he is a Hongshen Faculty with Chongqing University. He is the leader of Perception Group, Institute of Robot and Intelligent System (RIS-lab). His research interests include dexterous robotic manipulation and robotic perception.

Personal Homepage: https://orcid.org/0000-0002-8877-8524

---

---