A packaging robot, working in a candy factory in the near future, aims its electronic eye at a tray crammed with chocolates. The robot must fill a box with a specific mix of candy, but how can it tell a cream-filled morsel from a chocolate-covered peanut cluster?
A new object recognition technique developed at The Johns Hopkins University might help this robot hold onto its job. Using three lights, a video camera and a computer, researcher Elli Angelopoulou has devised a new way to transform visual images into electronic "signatures." Each is a distinctive series of 11 numbers. By comparing these signatures, a computer can tell how closely two objects resemble one another. For instance, a smooth, dome-shaped candy and a bumpy nut cluster would have far different signatures.
Angelopoulou, a doctoral student in the Department of Computer Science and an admitted chocoholic, has used the technique to identify coffee mugs, disposable razors, children's toys--and, yes, chocolate candies. It can even distinguish between two objects that have identical shapes but different colors. Her technique is primarily for recognizing objects with curved surfaces, a task that has long troubled computer vision scientists. The next step, Hopkins researchers say, is for robotics experts to incorporate Angelopoulou's breakthrough into a mechanical packaging machine. "As part of an industrial object recognition system, this technique could expand a robot's capability," says Larry Wolff, an associate professor of computer science who supervises the Computer Vision Laboratory in Hopkins' Whiting School of Engineering. Wolff and James P. Williams, a doctoral student who also is a member of the Computer Vision Laboratory group, contributed to the development of this new object recognition system.
A mechanical device that interacts with its environment must have some sort of visual information about its surroundings, says Angelopoulou (pronounced ann-jell-oh-POOH'-looh). "A robot or a vehicle that moves by itself needs to 'see' where it's going," she says. "We need a way for it to recognize objects." Scientists have invented electronic ways to identify some objects the way people often do: by following the edges until a rectangular shape, for example, is recognized as a door. But such systems do not work well on curved objects, such as vases, that have few edges to follow.
In 1995, Angelopoulou decided on a different approach: measuring the light that bounces off an object and into the lens of a video camera. "My goal was to find an object recognition technique by looking at the intensity values," she says. "That meant I was not going to look at the edges. I was going to look at every pixel and the value the camera recorded. I believed I could collect enough information for an object signature just by looking at the intensity values recorded by the camera under three lighting conditions."
Two years later, here's how her system works: Angelopoulou aims one lamp in the same direction as her camera lens and positions the other two on opposite sides of an object such as a coffee mug. She shoots the mug three times, first switching one lamp on and off, then repeating the process with the other two. The camera captures this mug in about 10,000 pixels--the tiny points on a video screen that blend together and form an image. For each pixel, Angelopoulou's software computes a DeCov value, short for determinant of the photometric covariance matrix. These values provide critical information about how the object curves. Within a few minutes, the computer sorts and reduces these values to a distinctive series of 11 numbers. These become the signature for the mug. If another picture produces the same or nearly the same series of numbers, the computer "recognizes" that object, too, as a coffee mug. Different colors reflect light differently, so the system can also separate blue and red mugs that have the same shape.
This system is less sensitive to the electronic "noise" or interference that hampers other computer vision systems, Angelopoulou says. And, unlike those systems, it does not require extremely precise placement and measurement of lighting. But the process does have limitations. It will recognize a mug from many upright angles, but the computer will be fooled if the camera looks at the flat bottom of the cup. Also, the computer will recognize identical objects only if the lighting remains unchanged.
A factory could provide such controlled lighting, and Angelopoulou hopes other scientists will someday install her system in a robot that could sort and pack mechanical parts, toys or even chocolates. "Usually, what we do in smaller labs is just try to solve a small part of the problem," she says. "Then we hope someone else will integrate it into a device that solves a bigger part of the problem."
Note:A black-and-white or color image is available to reporters on request. Contact Phil Sneiderman at the phone number or e-mail address above.