Ahmet Ekin, inventor of the new software called the Automatic Sports Video Analyzer (ASVA), envisions the program used in several different ways, such as creating edited versions of a game recorded on a personal video recorder like TiVo, or allowing a quick review of a game showing only plays where points were scored--or even watching two live games at once, with the plays of one game shown during the boring parts of the other. Ekin's advisor, A. Murat Tekalp, professor of electrical and computer engineering, even foresees ASVA as an economical way to bring sports to video cell phones.
Perhaps the greatest strength of ASVA is that it can be easily tailored to different aspects of different sports. For instance, a soccer match usually consists of very long plays with few breaks. The software can detect goals and replay those for you. Or a three-hour tennis match can become an hour of constant plays sans ball boys. Basketball gets a slighter trimming since it tends to have constant action, but the software could allow a viewer to watch an entire game of just one team's points.
Ekin's three-year venture started out as a part of his doctoral project. His goal was to get a computer to automatically understand what is going on in a piece of video. He chose video of sportscasts because they have well-defined events and objects. A touchdown, for instance, is an event that is usually pretty clearly denoted by crowd cheers, change of teams, instant replays and a longer-than-usual pause before play resumes. Certain images, like that of the quarterback, are fairly simple to recognize and stay relatively consistent throughout the length of the video. These two key elements allowed Ekin to test his algorithms and eventually achieve exceptional accuracy when differentiating between plays and time-between-plays.
"While developing the program, I tested it during last year's World Cup games," says Ekin. "Because they were held in Korea and Japan, the games were on before dawn. As a big soccer fan, I didn't want to miss anything but I sometimes couldn't afford to stay awake all night, so I captured videos of some of the games and used the software to give me summaries in the morning before the local TV could. It also let me get some sleep."
The software looks for changes in camera movement and sound to clue it in to what's happening on the field. In football, for instance, the view tends to stay at a wide angle, taking in both teams at once; then, as the play progresses, the cameras move and zoom in and out until the play ends. Then the view usually becomes close-ups of players catching their breath, coaches yelling from the sidelines, instant replays, and statistics on the game. The camera rarely returns to the wide-field view until the next play is about to begin. ASVA can monitor this camera movement in real time and tell with surprising accuracy when a play is about to begin. It works in real-time, which means that it doesn't have to backtrack to compare images or pause while it calculates.
The real-time capability means the software can pull off a nearly unimaginable feat--watching two live games at once.
ASVA would essentially watch the two games itself, monitoring their plays. It would display the play of one game while it watches the other, waiting for its boring bits to end. If game two is about to start a play and game one's play is finished, the software switches to the play of the second game. Obviously, the plays and lulls of each game are not likely to line up so neatly, but ASVA could pause plays while it waits to display them. At various times during a dual-viewing, certain plays from either live game may be delayed a few seconds at most. "It would be like picture-in-picture only you don't waste time and screen space watching the coaches spit," says Ekin.
Video over mobile phones is expected to be a giant market in the coming years and software like Ekin's would be exceptional at saving precious cellular bandwidth. By broadcasting only the plays of live games, half the bandwidth could be spared, meaning smaller cell bills for customers and fewer bandwidth headaches for suppliers.
One unexpected effect resulted from Ekin's work. He found that he and his thesis advisor are intense soccer fans; Tekalp's favorite team is Fenerbahce and Ekin's is Galatasaray. Ironically, both hail from Istanbul and are ancient rivals.
Ekin and Tekalp have applied for a patent on the offspring of Ekin's doctoral project and they hope to find a partner in industry to help them develop it further.