An online algorithm devised by Disney Research enables filmmakers to determine the optimal number and location of cameras for capturing a given scene, enabling them to amass the data that they increasingly want and need during post-production.
With sufficient information about the billions of light rays emanating from a scene, for instance, filmmakers can use computational techniques to relight the set virtually, to generate new viewpoints and to digitally recover the detailed 3-D shapes and layout of a scene, or even of an actor's face.
But knowing where to place cameras can be tricky, said Changil Kim, a Ph.D. student in computer graphics at Disney Research and ETH Zurich. If cameras are few and far between, the chances that a location will be blocked by an object in the foreground are increased. And if cameras are too close together, the differences in successive views can become too tiny to provide any value or useful information.
"To solve this, we focused on the problem of automatic online view-sampling, where each new view results in the best possible additional shape reconstruction incrementally," Kim said. At each iteration, the algorithm he and his colleagues developed adds a camera view and calculates a depth map of a scene; this process continues until the last camera view added results in a negligible improvement.
If there is a limit on the number of cameras available, the algorithm can stop at that number, providing an optimal spacing and placement of that number of cameras.
The Disney researchers presented their findings at the International Conference on Image Processing (ICIP 2015) in Quebec City. Conference organizers have recognized their report as among the top 10 percent of papers being presented this year.
The researchers compared their new method with regular sampling on both synthetic and real scenes and showed they could recover high quality detail with fewer than half the number of cameras required for regular sampling.
"Our online sampling analysis considers the statistics of the very scene being captured and thus results in a significant win over dense, arbitrarily or regularly placed cameras," said Kenny Mitchell, a senior research scientist at Disney Research.
###
In addition to Kim and Mitchell, the research team included Alexander Sorkine-Hornung and Markus Gross of Disney Research and Kartic Subr of Heriot-Watt University in Edinburgh, Scotland. For more information about the project, visit the project page at: http://www.disneyresearch.com/publication/online-view-sampling/
This research is part of a collaborative project including teams from Industrial Light and Magic in London and Vicon Motion Systems in Oxford and is partly supported by Innovate UK. The project, Real-Time Digital Acting, seeks to improve the competitiveness of the United Kingdom's computer graphics industry by revolutionizing production pipelines for films and video games.
About Disney Research
Disney Research is a network of research laboratories supporting The Walt Disney Company. Its purpose is to pursue scientific and technological innovation to advance the company's broad media and entertainment efforts. Vice Presidents Jessica Hodgins and Markus Gross manage Disney Research facilities in Los Angeles, Pittsburgh, Zürich, and Boston and work closely with the Pixar and ILM research groups in the San Francisco Bay Area. Research topics include computer graphics, animation, video processing, computer vision, robotics, wireless & mobile computing, human-computer interaction, displays, behavioral economics, and machine learning.