Sternberg, of Yale University, has conducted two studies, using experimental tests to supplement the Standardized Admission Test (SAT) and the Graduate Management Admission Test (GMAT). His results were presented during a AAAS session entitled "Testing and University Admissions," at 9:00 a.m., Eastern Time, on Saturday, 16 February.
The tests are based on Sternberg's theory that intelligence consists of three components. Analytical ability, which is measured by the SAT, Graduate Record Examination (GRE), and similar tests, involves analyzing, comparing and contrasting, evaluating, and critiquing, according to Sternberg. Creative abilities include inventing, discovering, supposing and imagining, while practical abilities include putting ideas into practice, using newly-learned information, and putting things into a "real world" context.
Neither creative nor practical abilities are measured by typical standardized tests, said Sternberg , so he and his colleagues devised new tests that do.
"Many people feel like the time has come for the testing industry to change," Sternberg said. "I think this is going to be the future of testing."
Standardized admissions testing has indeed come under more scrutiny than usual lately. The President of the University of California at Berkeley, for example, has questioned the university's use of the SAT(I), the test most widely used in the admissions process. Meanwhile, competition to attend selective colleges and universities has never been more intense.
"The competition should be over the right things. But students are now competing for a very limited range of things," Sternberg said.
Sternberg and his colleagues conducted their experiments with students at 16 different high schools and colleges, and graduate students at the University of Michigan business school. The researchers gave the students tests based on all three types of skills, and compared their scores with the grades and other evaluations the students later received during their college or graduate education.
The results indicated that adding the new tests to the SATs or GMATs should substantially improve the tests' ability to predict student success, beyond what the standardized tests predicted by themselves. Differences between how well different racial or ethnic groups scored were also reduced in the new tests, compared to the SAT and GMAT, according to Sternberg.
The researchers plan to complete another round of both studies, revising them slightly and expanding the number of students who take them.
Sternberg and his colleagues' investigation of the SAT, funded by The College Board, is called the "Rainbow Project." Its measure of practical abilities included a multiple choice section involving planning routes, solving practical everyday reasoning problems, and using practical math. This part of the test also involved presenting students with various school- or office-related problems. The students then had to propose possible solutions themselves, or rate a series of solutions, from excellent to awful.
The creative test also includes multiple-choice questions, involving working in new numeration systems for example, or solving reasoning problems with counter-intuitive presuppositions. Students also did performance-based exercises, such as telling stories based on captioned cartoons, or using software to design company logos.
In a study funded by the University of Michigan, Sternberg and his colleagues posed two types of problems to MBA students. The first involved a variety of information about a complex business problem, in the form of data charts, emails, letters, and so forth. Students had to figure out what the problem was, how to solve it, and how to evaluate whether the solution worked. The second part of the test described problems more briefly, and proposed a list of possible solutions. Students then rated the solutions.
While these tests can be conducted over the Internet, scoring them would be more labor-intensive than scoring all-multiple choice tests like the SAT. The subjective sections of the Rainbow Project were scored by trained graders, while the subjective sections of the University of Michigan test were scored according to how successful business people answered the questions.
Sternberg thinks the testing industry will be willing to make the extra effort, however. "The challenges to conventional tests are getting stronger," he said. "Organizations that don't adapt won't survive."
The American Association for the Advancement of Science (AAAS) is the world's largest general scientific society, and publisher of the journal, Science. Founded in 1848, AAAS serves 134,000 members as well as 273 affiliates, representing 10 million scientists.