UConvTrans: A new method for cardiac image segmentation

Researchers from University of Science and Technology Beijing have proposed a new method for cardiac image segmentation. The work, published in Journal of Shanghai Jiao Tong University, showcased a model called UConvTrans, which is a dual-flow cardiac image segmentation network characterized by global and local information integration.

Magnetic resonance imaging (MRI) is a widely-used medical imaging technique which helps examine organs and tissues in human bodies and assists the diagnosis of various diseases including cardiac diseases. Segmentation of magnetic resonance images is an essential step in the analysis of MRI results. The traditional U-shaped convolutional neural networks (CNN) structure is able to handle most cases of medical image segmentation. However, it can hardly perform well in cardiac magnetic resonance image segmentation as there is much noise, the target areas are indistinguishable from the background and the shape of the right ventricle is irregular.

“Due to its local operation characteristics, the ability of convolution for global feature extraction is limited,” explained the first author Professor Qing Li from school of automation and electrical engineering, University of Science and Technology, who led the research. “That is the main problem we have to conquer when building a new model which aims to improve the accuracy of cardiac magnetic resonance image segmentation.”

Yubin Huangfu, the second author of the article, shared more details of model building. “The Transformer structure can capture global representations, which may compensate for the defects of CNN. So, we decided to build a new model which combines CNN and Transformer. After fully investigating the model structure design ideas of image segmentation task based on Transformer architecture in the field of general natural images and the necessary factors required to realize precise medical image segmentation, we designed the preliminary baseline for testing,” he told. “According to the characteristics of medical image segmentation, we introduced a model design that captures global dependencies and local details to improve the model performance. Then, we introduced various effective lightweight methods to improve the performance and efficiency of the model while ensuring that the accuracy remains unchanged as much as possible. That is how UConvTrans comes into being.”

UConvTrans processes a strong ability of extracting both local and global features owing to the introduction of Transformer into traditional CNN. The CNN branch extracts local features while the Transformer branch obtain global representations, which retains local detailed features while suppressing the interference of noise and background features. A bidirectional fusion module called Fuse CNN and Transformer Block (FCTB) was proposed by the researchers to fuse the features extracted by CNN and Transformer with each other, which in turn enhances the feature expression capability and improves the segmentation accuracy of the right ventricle. Furthermore, parameter setting of UConvTrans becomes flexible as the Transformer structure in the model does not require pre-trained weights.

UConvTrans is tested on the MICCAI 2017 ACDC dataset including short axis cardiac MRI of 150 patients, which proves that the model strikes a better balance between precision and efficiency compared with existing models based on CNN or Transformer. Particularly, UConvTrans outperforms the traditional U-Net by 1.13% average dice coefficient while the parameter amount and the floating point operations are only 10% and 8% of the U-Net. UConvTrans eventually achieves a dice coefficient 92.42% for the right ventricle, 91.64% for the myocardium and 95.06% for the left ventricle respectively, which ranks first in the myocardium left ventricle on the test.

“UConvTrans is able to conduct accurate segmentation of target areas in cardiac MRI and it outperforms the existing models in terms of efficiency, suggesting a wide application prospect in cardiac image segmentation,” said Professor Jiangyun Li, the corresponding author of the work. “However, there is still much to do to fully tap the potential of the model. First, the current UConvTrans is a 2D network which handles 2D images only. Therefore, it needs further improvement to be capable of handling 3D medical images with more information. In addition, the efficiency of the model needs further improvement so that it can be deployed to terminal systems which require real-time performance. Furthermore，evaluation of the generalization of UConvTrans is required to know how the model performs on the medical image data of other organs and tissues.”