To advance research into open-source software for programmable cameras, Stanford photography researchers are taking an unusual approach – giving away cameras to scientists at other universities.
Armed with a $1 million grant from the National Science Foundation (to be shared with colleagues at MIT,) Stanford will produce professional-style, single-lens reflex (SLR) cameras, equipped with the "Frankencamera" open-source software platform, for free distribution to computational photography professors around the country.
Non-academics may buy the camera at cost. Plans call for the cameras to be available within a year.
Going down a second avenue, the research team has struck an agreement with Nokia, the cell phone company. Frankencamera, which allows users to create novel camera capabilities, is now available as a free download for Nokia N900 "mobile computers."
"We're going public with Frankencamera," said Stanford computer science and electrical engineering professor Marc Levoy, who leads the project. "We are releasing code so that people can create new imaging applications on their Nokia N900s."
Program and shoot
Next week at the SIGGRAPH conference in Los Angeles, the Frankencamera engineering team will describe the platform and several sample apps created with it.
Computational photography refers to the ways computers can extend the capabilities of digital imaging by combining multiple photographs taken with different camera settings to create an image that could not be taken in a single shot, or with an ordinary camera.
Some of these new ways of combining images can be done in Photoshop or another such program, but until now they could not be done inside the camera, Levoy said. That's because commercial cameras are closed to development by all but their manufacturers. Frankencamera, on the other hand, brings computational photography directly to the camera, by making the camera a programmable platform.
Frankencamera began in 2006 when Levoy and Kari Pulli, a Nokia Fellow who heads a research team at Nokia Research Center (NRC) Palo Alto, and a former research associate in Levoy's lab, reasoned that computational photography shouldn't be relegated to klunky research equipment in academic labs, as it has been for years. Instead it should be developed for use in the field on portable, consumer-friendly cameras.
"We thought it was time to make the research more nimble and get the results into smaller form-factors," said Pulli. "At NRC, we believe in an open innovation model that enables consumers and university researchers to use our research algorithms, and add their own, to create even more interesting capabilities."
At SIGGRAPH the researchers will describe how the Frankencamera platform exposes all the photographic and computational hardware on the camera – light sensors, flashes, focus, shutters and image processors – to a programmer's control through a software interface. The interface is part of a software "stack" that brings together familiar programming elements: a Linux operating system and the ubiquitous C++ programming language.
"The N900 is a camera phone, but it runs a version of Linux almost as complete as that installed on personal computers," Pulli said.
Putting the "app" in aperture
To help demonstrate and inspire development with Frankencamera, the team will discuss six apps they've created on the platform. Using a prototype of their SLR version, for example, Levoy's group hooked two flashes to the camera and programmed one to blink like a strobe light and the other to fire at the end of the strobe sequence. Then they flung some playing cards in the air. The result was a shot that captured the cards' trajectories and also caught them still in mid-air with remarkable clarity.
Fine control of multiple flashes may yield captivating art, but sometimes photographers are caught with terrible lighting conditions that a flash can't solve. Confronted with inadequate light, a photographer with a conventional camera must choose either a quick exposure that will look sharp, but dark and noisy, or a long exposure that will have enough light, but will likely be blurry.
An app called "low-light imaging" shoots both exposure speeds in rapid succession and then automatically combines them, resulting in a photo that is both bright and sharp.
Another app offers a different take on the low-light problem. Dubbed "lucky imaging," the app takes advantage of a motion sensor attached to the Nokia N900 to prevent shaky hands from adding blur to a long exposure. In this mode, the camera shoots constantly but only stores images taken when the camera is at its most still. The result is the sharpest possible image.
Levoy said he hopes the entire computational photography community will step forward with other ideas and that they will share them with each other online. The apps will be found through the Frankencamera project website.
For Levoy, a key question is whether commercial manufacturers – both of dedicated cameras and camera smart phones – will also open up their platforms. Since the Frankencamera effort was first publicized last fall, Levoy said, he has had a steady stream of meetings, especially with smart phone makers.
"What we're hoping is that if making your camera programmable adds value to the camera that this could shift the entire camera industry," Levoy said. "We'd like to democratize cameras the way that the Maker Faire is democratizing crafts so that anyone can play the game."
In addition to Nokia Research Center, funding comes from Adobe Systems, Kodak, Hewlett-Packard and the Walt Disney Company. In addition to authors at Stanford and NRC, other authors on the SIGGRAPH paper are affiliated with Ulm University in Germany, the University of California-Santa Barbara and Disney Research in Zurich, Switzerland.
"We're going public with Frankencamera," said Stanford computer science and electrical engineering professor Marc Levoy, who leads the project. "We are releasing code so that people can create new imaging applications on their Nokia N900s."
In addition, the researchers have been awarded a $1 million grant from the National Science Foundation, shared with colleagues at MIT, to begin making professional-style, single-lens reflex (SLR) cameras, equipped with the software platform, for free distribution to computational photography professors around the country. Non-academics could buy the camera at cost. Levoy said he expects those cameras will be available within a year.
To help demonstrate and inspire development with Frankencamera, the team will discuss six apps they've created on the platform. Using a prototype of their SLR version, for example, Levoy's group hooked two flashes to the camera and programmed one to blink like a strobe light and the other to fire at the end of the strobe sequence. Then they flung some playing cards in the air. The result was a shot that captured the cards' trajectories and also caught them still in mid-air with remarkable clarity.
Fine control of multiple flashes may yield captivating art, but sometimes photographers are caught with terrible lighting conditions that a flash can't solve. Confronted with inadequate light, a photographer with a conventional camera must choose either a quick exposure that will look sharp, but dark and noisy, or a long exposure that will have enough light, but will likely be blurry.
An app called "low-light imaging" shoots both exposure speeds in rapid succession and then automatically combines them, resulting in a photo that is both bright and sharp.
Another app offers a different take on the low-light problem. Dubbed "lucky imaging," the app takes advantage of a motion sensor attached to the Nokia N900 to prevent shaky hands from adding blur to a long exposure. In this mode, the camera shoots constantly but only stores images taken when the camera is at its most still. The result is the sharpest possible image.
Levoy said he hopes the entire computational photography community will step forward with other ideas and that they will share them with each other online. The apps will be found through the Frankencamera project website.
For Levoy, a key question is whether commercial manufacturers – both of dedicated cameras and camera smart phones – will also open up their platforms. Since the Frankencamera effort was first publicized last fall, Levoy said, he has had a steady stream of meetings, especially with smart phone makers.
"What we're hoping is that if making your camera programmable adds value to the camera that this could shift the entire camera industry," Levoy said. "We'd like to democratize cameras the way that the Maker Faire is democratizing crafts so that anyone can play the game."
In addition to Nokia Research Center, funding comes from Adobe Systems, Kodak, Hewlett-Packard and the Walt Disney Company. In addition to authors at Stanford and NRC, other authors on the SIGGRAPH paper are affiliated with Ulm University in Germany, the University of California-Santa Barbara and Disney Research in Zurich, Switzerland.