Virtual fitting room

A team of researchers have created a way for people to visualize themselves wearing items of clothing they don’t have direct physical access to. The virtual try-on system uses a unique capture device and an artificial intelligence (AI)-driven way to digitize items of clothing. Using a corresponding imaging and display system, a user can then view themself on a screen wearing anything from the digital wardrobe. An AI engine synthesizes photorealistic images, allowing movement and details such as folds and ripples to be seen as if the user is really wearing the virtual clothing.

Shopping for clothes can be difficult sometimes. Often you get to the store and they’re out of the style or size of garment you intended to try on for yourself. A lot of shopping can be done online now, but shopping for clothes can be difficult as it’s very hard to judge how well something will fit you or whether you’ll like it once you see it in person. To address these issues, there have been some attempts over the years to create a sort of digital mirror that can visualize a user and superimpose an item of clothing on top of them. However, these systems lack functionality and realism.

Professor Takeo Igarashi from the User Interface Research Group at the University of Tokyo and his team explore different ways humans can interact with computers. They felt they could create their own digital mirror that resolves some limitations of previous systems. Their answer is the virtual try-on system, and the team hopes it can change how we shop for clothes in the future.

“The problem of creating an accurate digital mirror is twofold,” said Igarashi. “Firstly, it’s important to model a wide range of clothes in different sizes. And secondly, it’s essential these garments can be realistically superimposed onto a video of the user. Our solution is unique in the way it works by using a bespoke robotic mannequin and a state-of-the-art AI that translates digitized clothing for visualization.”

To digitize garments, the team designed a mannequin that can move, expand, and contract in different ways to reflect different body poses and sizes. A manufacturer would need to dress this robotic mannequin in a garment, then allow it to cycle through a range of poses and sizes while cameras capture it from different angles. The captured images are fed to an AI that learns how to translate them to fit an as yet unseen user. At present the image capture for one item takes around two hours, but once someone has dressed the mannequin, the rest of the process is automated.

“Though the image capture time is only two hours, the deep-learning system we’ve made to train models for later visualization takes around two days to finish,” said graduate student Toby Chong, one of the paper’s co-authors. “However, this time will naturally decrease as computational performance increases.”

Then comes the user interaction. Someone wishing to try on different clothes would need to come to the store and stand in front of a camera and screen. They would put on a specially made outfit with an unusual pattern on it called the measurement garment. The pattern is a nonrepeating arrangement of different colored squares. The reason for this is it is easy for a computer to estimate how someone’s body is positioned in space according to how these colored squares move relative to one another. As the user moves, the computer synthesizes a plausible image of the garment that follows the user's motion.

“An advantage of our system is that the virtual garment matches the motion of the measurement garment including things like wrinkles or stretching. It can look very realistic indeed,” said Igarashi. “However, there are still some issues we wish to improve. For example, at present the lighting conditions in the user’s environment need to be tightly controlled. It would be better if it could work well in less-controlled lightning situations. Also, there are some visual artifacts we intend to correct, such as small gaps at the edges of the garments.”

The team has several possible applications in mind, online shopping being the most obvious, with a side benefit being it could prevent a large amount of fashion waste as buyers could have greater assurance about their purchase and clothing could even be made on demand rather than at scale. But virtual try-on could also find use in the world of online video, including conferences and production environments, where appearance is important, and budgets are tight.

 

###

Journal article
Toby Chong, I-Chao Shen, Nobuyuki Umetani, and Takeo Igarashi, “Per Garment Capture and Synthesis for Real-time Virtual Try-on” ACM Symposium on User Interface Software and Technology (UIST ’21), DOI: 10.1145/3472749.3474762
https://doi.org/10.1145/3472749.3474762

 

Funding
This work was supported by JST CREST Grant Number JPMJCR17A1, Japan. I-Chao Shen was supported by JSPS KAKENHI Grant Number JP21F20075.

Links
User Interface Research Group

https://www-ui.is.s.u-tokyo.ac.jp/en/

Department of Creative Informatics

https://www.i.u-tokyo.ac.jp/edu/course/ci/index_e.shtml

Graduate School of Information Science and Technology

https://www.i.u-tokyo.ac.jp/index_e.shtml

 

Research Contact

Professor Takeo Igarashi

Graduate School of Information Science and Technology, The University of Tokyo

7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, JAPAN

Email: takeo@acm.org

 

Press Contact

Mr. Rohan Mehra

Division for Strategic Public Relations, The University of Tokyo

7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8654, Japan

Email: press-releases.adm@gs.mail.u-tokyo.ac.jp

 

About the University of Tokyo

The University of Tokyo is Japan's leading university and one of the world's top research universities. The vast research output of some 6,000 researchers is published in the world's top journals across the arts and sciences. Our vibrant student body of around 15,000 undergraduate and 15,000 graduate students includes over 4,000 international students. Find out more at www.u-tokyo.ac.jp/en/ or follow us on Twitter at @UTokyo_News_en.

---

---