Modern films and TV shows are filled with spectacular, computer-generated sequences which are computed by rendering systems that simulate the flow of light in a 3D scene. However, computing many light rays is an immensely labor-intensive and time-consuming process. The alternative is to render the images using only a few light rays, but this shortcut results in inaccuracies that show up as objectionable noise in the final image.
Researchers from Disney Research, Pixar Animation Studios, and the University of California, Santa Barbara have developed a new technology based on artificial intelligence (AI) and deep learning that eliminates this noise and thereby enables production-quality rendering at much faster speeds.
Specifically, the team used millions of examples from the Pixar film Finding Dory to train a deep learning model known as a Convolutional Neural Network. Through this process, the system learned to transform the noisy images into noise-free images that resemble those computed with significantly more light rays. Once trained, the system was successfully able to remove the noise on test images from entirely different films, such as Pixar's latest release, "Cars 3," and their upcoming feature "Coco," even though they had completely different styles and color palettes
"Noise is a really big problem for production rendering," said Tony DeRose, head of research at Pixar. "This new technology allows us to automatically remove the noise while preserving the detail in our scenes."
The work presents a significant step forward over previous, state-of-the-art denoising methods which often left artifacts or residual noise that required artists to either render more light rays or to tweak the denoising filter to improve the quality of a specific image. Disney and Pixar plan to incorporate the technology in their production pipelines to accelerate the movie-making process.
"Other approaches for removing image noise have grown increasingly complex, with diminishing returns," said Markus Gross, vice president for research at Disney Research. "By leveraging deep learning, this work presents an important step forward for removing undesirable artifacts from animated films."
The work will be presented in July at the ACM SIGGRAPH 2017 conference, the premier venue for technical research in computer graphics. To facilitate further exploration of this exciting area, the team will make their code and trained weights available to the research community.
###
About UCSB
The College of Engineering at UC Santa Barbara is consistently ranked among the upper echelon of engineering schools in the world. It provides students with the direct academic mentorship they need to complete degree programs on time and build successful careers. The college is built on collaborative interdisciplinary innovation, and faculty and students are committed to developing new technology that improve the world and add economic value in our region, our state, and around the world. Having developed one of the most successful public-private research partnership environments in the nation, UCSB is also a hotbed of new intellectual property.
About Disney Research
Disney Research is a network of research laboratories supporting The Walt Disney Company. Its purpose is to pursue scientific and technological innovation to advance the company's broad media and entertainment efforts. Vice President Markus Gross manages Disney Research facilities in Los Angeles, Pittsburgh and Zürich, and works closely with the Pixar and ILM research groups in the San Francisco Bay Area. Research topics include computer graphics, animation, video processing, computer vision, robotics, wireless & mobile computing, human-computer interaction, displays, behavioral economics, and machine learning.