image: This shows Professors Matthew Turk (left) and Tobias Hollerer. view more
Credit: University of California - Santa Barbara
Augmented reality applications for mobile devices could become smarter and more sophisticated, thanks to two recent grants awarded to UC Santa Barbara computer science professors Matthew Turk and Tobias Höllerer.
While many mainstream augmented reality (AR) applications rely on mobile device sensors and a static dataset layered over real-time visuals or GPS coordinates, Turk and Höllerer envision next-generation AR that is more stable, realistic, and dynamically updated by users.
"Our research employs real-time computer vision for more stable presentation of 3D computer graphics that appear as if they are truly part of the physical world," said Professor Höllerer. "Imagine applications, such as a landscape architect experimenting with design by placing virtual trees or walking within the grounds they plan to develop. A tourist at an archaeological site could explore the reconstruction of an ancient temple wh ere it once stood."
The UCSB team is conducting intensive research that would couple mobile computer vision capture with crowdsourced user data that could immediately discern whether the app object matches the object in reality. They've termed it "anywhere" augmented reality.
There is a need to close the sensor loop to get more accurate overlays, Höllerer explained. "The camera feed from the mobile device can be matched to indexed image databases. But images of the real world change. So, the next time I'm standing in front of a restaurant using an AR app, and the façade has recently changed, I can update that information just by virtue of looking at it. The use of the app improves the experience of the next user."
To achieve "anywhere augmentation" the researchers must first design an interface with which potential developers can easily experiment out of the box.
With a recent $300,000 grant from the federal Office of Naval Research, Turk and Höllerer's computer science research group—called the Four Eyes Lab—will be closely studying user experience to create that optimal user interface for AR. The project is a collaboration with Virginia Tech Professor Doug Bowman.
Höllerer and Bowman turned to the virtual reality world for testing and prototyping, using head-worn gear and tablet computers in the UCSB's AlloSphere Research Facility—a unique 3-story immersive virtual reality laboratory housed in the UCSB NanoSystems Institute. The AlloSphere provides a seamless 180-degree, three-dimensional environment integrated with advanced computer equipment for complete data visualization immersion.
Their research group was also recently awarded $500,000 over three years from the National Science Foundation for a project that uses computer vision-based tracking and augmented reality to enhance remote collaboration (telecollaboration) in physical spaces.
"Live teleconferences would mimic in-person meetings where an engineer or doctor, for example, would be able to interact remotely with a three-dimensional space, such as an instrument or with a team of people," explained Höllerer. Their project allows users in different locations to see and assign data within a target scene, extending two-dimensional tracking to a real-time three dimensional scenario.
"You can point to and annotate an object in a target environment through your screen, and the annotation will 'stick' to the object even when the camera moves, and it will be visible to all users," said Turk. Prototype testing of their technology allowed users, instructed by a remote expert, to control a mock-up airplane cockpit using just a visual camera feed.
Next-generation AR could make it possible to explore any physical environment, known or unknown, live from a remote location, according to the researchers. Rapid advancements in mobile computing devices in recent years—primarily smartphones and tablets—provide a perfect springboard for the technology. "The applications for mobile, real-time augmented reality can have a major impact on health, education, entertainment, and many other areas," added Turk.
Advancements in augmented reality from the UCSB Four Eyes Lab have caught the attention of the global research community this past year. Their work on live tracking and mapping recently won the best paper award at the selective 2012 International Symposium on Mixed and Augmented Reality (ISMAR).
Research at the UCSB Four Eyes Lab focuses on the "four I's" of Imaging, Interaction, and Innovative Interfaces. The laboratory is directed by professors Matthew Turk and Tobias Höllerer and is affiliated with the UCSB Department of Computer Science and the Media Arts and Technology graduate program.
The Department of Computer Science at UC Santa Barbara is part of the College of Engineering, a recognized global leader among the top tier of engineering education and research programs.