Meet Steve and Adele, pedagogical agents from the ISI.
Steve and Adele are advanced softbots who facilitate human learning -- robots consisting solely of software and employing the latest in artificial intelligence technology from the Information Sciences Institute (ISI) at the University of Southern California's School of Engineering.
Steve (Soar Training Expert for Virtual Environments) hovers nearby as you move electronically through a maze of controls in the engine room of a virtual U.S. Navy surface ship. He provides explanations, answers questions, gives demonstrations, and offers helpful hints when you're stumped. He never makes a mistake and never tires; yet he has infinite patience with human fallibility and fatigue.
"Steve is completely autonomous and supports human learning in a highly complex environment," says Lewis Johnson, Ph.D., an ISI research associate professor and director of the institute's Center for Advanced Research in Technology for Education (CARTE). "Steve is programmed with a family of problem-solving strategies, or sets of possible behaviors, enabling him to take advantage of learning opportunities as they arise. He's not merely programmed to follow a fixed script. He can make decisions as a task is being performed."
To meet Steve electronically, you first must don a head- mounted display -- a helmet containing tiny computer screens in front of each eye -- and a pair of data gloves with built-in position and touch sensors. You are now equipped to enter the stereoscopic, three-dimensional environment of the virtual engine room he inhabits. Your view and perspective will change as you turn your head or move through the virtual room. You can watch from multiple angles as Steve demonstrates a task and explains it to you verbally.
"I will now perform a functional check of the temperature monitor to make sure that all of the alarm lights are functional," Steve intones in a clearly synthetic voice while looking into your eyes. As he speaks, Steve reaches with one hand and presses the function test button, turning on a row of lights. He continues to speak as he points to the lights. "All of the alarm lights are illuminated, so they are all working properly."
Steve can work with several students -- representing the ship's crew -- at once. He understands how the crew members' roles interact. Moreover, he can coordinate his instruction with other virtual agents who can assist individual "crew members" or play the role of a missing seaman.
Steve has three main software modules for perception, cognition and motor control, Dr. Johnson explains.
The perception module keeps track of the current state of the virtual world. It receives notifications from the human/computer interface controlling the student's display as the student moves and performs actions, and from the software that simulates the functions of the virtual ship. It is also notified whenever students or other agents speak to each other. Equipped with speech recognition software, Steve is able to understand within the limitations of his programming.
The perception module uses this information to maintain a model of what is happening in Steve's environment. Steve's cognitive module refers to this model when making decisions about what to do next.
When the cognitive module decides to take an action, it transmits a command to the motor control module, which responds by moving the agent's body through the virtual world.
The cognitive module includes general pedagogical software enabling the softbot to monitor students, explain actions and demonstrate tasks. For each subject he teaches, Steve is given a set of task plans. These task plans do not prescribe a fixed ordering of steps or specify how the task should be taught. A single task plan allows Steve to demonstrate the task, coach the student as he or she practices it, and answer questions about it.
Adele (Agent for Distance Education Learning Environments) is a two-dimensional, animated persona implemented as a Java-based applet for medical students taking on-line courses. An applet is a software program that is downloaded automatically off the World Wide Web and run on the student's computer.
In her white medical coat, carrying a clipboard and with a stethoscope draped around her neck, Adele monitors students as they examine a virtual patient on their computers.
If the student performs an action that's inconsistent with standard practice, Adele interrupts and suggests an action to perform instead.
Like Steve, Adele is programmed to instruct whenever she gets the chance. For example, she asks students in a clinical decision-making course to examine -- by using their computer mice -- a simulated patient's slow-growing lesion of the skin and to diagnose the disease suggested by the patient's range of symptoms.
Like Steve, Adele can be programmed to teach a variety of subjects. While her first assignment has been to instruct medical students and physicians, Johnson has already begun adapting Adele for dentistry coursework and is investigating the possibilities for an electrical engineering course.
Unlike Steve, who runs on a powerful Silicon Graphics workstation, Adele runs on a consumer-level personal computer. Much of the data needed for a particular course (such as a library of x-ray images) can be loaded onto students' computers ahead of time to minimize what must be sent from the web server during class.
Johnson says pedagogical agents help both students and teachers.
"Teachers are most effective when they work one-on-one with students, but human teachers can't work one-on-one with everyone in the class at once. Software agents can, and they can be available all the time," he says. "Agents give teachers more freedom to focus on important problems, because the agents can handle routine tasks like the grading of tests. Also they can emphasize course areas that often pose difficulties for students, and they can identify individual students who are having problems."
Johnson believes that software agents like Adele will enrich distance learning, making it both more interactive and engaging.
"Live lectures can be boring, and watching a boring lecture on a monitor is worse," he says. "Here, students are actively solving problems in an engaging environment and getting individual help."
A current research thrust is to enable a softbot to perceive human facial expressions, body language and visible emotions and react to them appropriately.
"Emotive behavior by the agent can help engage and motivate the learner," says Johnson, "and agents can alleviate student frustration by appearing to empathize with their problems. On the other hand, it's very tricky to decide when Adele or Steve should register disapproval of a student's failing efforts."
Johnson says Steve's voice sounds very artificial because it comes from an off-the-shelf, text-to-speech generator. "We didn't want to record a real voice because we often have to adjust what the agent says. Also, audio clips take up a lot of memory."
Nevertheless, Steve's body has been evolving to become more humanoid.
"He started out as just a head and torso with a disconnected hand, but his lack of arms bothered too many people," says Johnson. "So we have gradually been adding body parts to make him more lifelike. We have given him a complete upper body, including arms. He still has no legs, though, and probably won't get them. In the tight confines of the virtual reality view, you could scarcely see his legs anyway."
ISI computer scientist Jeff Rickel, Ph.D., is the co-developer of Steve, and the research is funded by the Office of Naval Research and the Air Force Research Laboratory. Steve's virtual ship environment was developed in collaboration with USC's Behavioral Technology Laboratory and Lockheed Martin.
Erin Shaw, a systems programmer at ISI, is the co-developer of Adele, and the research is funded by ISI and the USC Health Consortium.
EDITOR: Dr. Johnson is a resident of Venice, Calif.
For more information about Steve and Adele and the Center for Advanced Research in Technology for Education (CARTE), point your browser to http://www.isi.edu/isd/carte/.
For computer graphics of Steve and Adele, call Glenn Seki at 213-740-9335 or send email to gseki@usc.edu.