Public Release: 

Smart software makes sense of sketches

New Scientist

INTELLIGENT software that brings rough sketches to life in a virtual world is promising to revolutionise the way children learn and to help engineers visualise their designs. In designing the software, developers at the Massachusetts Institute of Technology have had to tackle several tough tasks. First the software must recognise crude hand-drawn shapes in the way the user intended - for example, by spotting that four wiggly lines represent a square with straight sides. Then it has to recognise the context of the objects being sketched, in the same way that we interpret two circles under a box on a slope as some kind of vehicle on wheels. Finally, the package has to animate the sketch so that objects move as they would do in the real world.

Current sketching software only recognises the geometry of a sketch - for example, that two lines meet at a specific angle - and cannot recognise context at all. The MIT software monitors the image as it is being drawn onto a computer screen and allocates probabilities to various interpretations of what it might represent. As the user adds more detail, the software adjusts these weightings. To do this, it uses a technique known as Bayesian analysis, which is normally used to compute the likelihood of specific causes, given certain effects.

"With our software, the 'causes' are what the user had in mind to draw, and the 'effects' are what was actually drawn," says Randall Davis, who has developed the code with Christine Alvarado at MIT's Artificial Intelligence Laboratory. Once the computer has settled on an interpretation, it applies the laws of physics such as gravity and friction to set the virtual world in motion. Cars roll down hills, pendulums swing, and objects bang into each other. The Bayesian part of the software is still at an early stage: it recognises only squares. Davis's next goal is to get it to recognise a much broader range of crudely drawn shapes. Part of the problem is thinking up all the possible ways someone might draw a shape, so as to be able to assign realistic probabilities to particular patterns on the screen.


Written by Celeste Biever

UK CONTACT - Claire Bowles, New Scientist Press Office, London:
Tel: 44-207-331-2751 or email

US CONTACT - Michelle Soucy, New Scientist Boston Office:
Tel: 617-558-4939 or email

New Scientist issue: 13th September 2003


"These articles are posted on this site to give advance access to other authorised media who may wish to quote extracts as part of fair dealing with this copyrighted material. Full attribution is required, and if publishing online a link to is also required. Advance permission is required before any and every reproduction of each article in full - please contact Please note that all material is copyright of Reed Business Information Limited and we reserve the right to take such action as we consider appropriate to protect such copyright."

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.