Dr. Eric Loken, Penn State assistant professor of human development and family studies who led the study says, "That study pattern will be especially problematic for students preparing for the 2005 SAT in which analogies will be eliminated and more reading comprehension questions and writing will be added."
The analysis is detailed in a paper, "Online Study Behavior of 100,000 Students Preparing for the SAT, ACT, and GRE," in the current issue of the Journal of Educational Computing Research. Loken's co-authors are Filip Radlinski, graduate student at Cornell University; Dr. Vincent H. Crespi, Penn State professor of physics; Dr. Josh Millet, president of www.number2.com, a division of XAP Corp., and Lesleigh Cushing, assistant professor of philosophy and religion, Colgate University.
The study data came from students who used a test preparation Web site, www.number2.com, launched by the study authors in September 2000. The students used the site between Dec. 8, 2001 and Dec. 8, 2002, including 55,015 preparing for the SAT, 22,318 for the ACT and 25,889 for the GRE.
Loken notes, "Tracking activity on the Internet provides unique information about study behavior. When students buy books or software, it is almost impossible to determine how they actually use the materials."
Peaks in preparation activity for the ACT and SAT matched the national administrations of the test. There were no GRE peaks because the test is administered on computer and no longer has national test dates. A significant percentage of students, especially high schoolers, enrolled on the Web site within two weeks of their test date.
GRE-takers, usually college age or older, spent more than twice as much time on the Web site than the ACT and SAT-takers who are usually high schoolers. The GRE students also invested more effort, as measured by page views and questions answered, than SAT and ACT students.
Fewer than 30 percent of the SAT and GRE students attempted any math questions. Only 20 percent attempted any reading comprehension even though those questions are half the total verbal score on the tests. The ACT is the only test that includes science questions but only 20 percent of ACT-takers practiced for the science section.
To see if the order of presentation of the study material affected what the students studied, the researchers randomly assigned students preparing for the SAT to a home page with either the math or verbal sections listed first for a seven-week period. They found that, when the math questions were listed first, the percentage of students who answered math questions rose from 28 percent to 38 percent but analogies were still the post-popular practice questions.
"Our study suggests that having a parent or teacher monitor the student's progress can help ensure that they practice math as well as verbal questions, for example," Loken notes. "Online tools make such monitoring possible."
He adds that some of the improvement in scores that commercial test preparers tout in their advertising may be due simply to encouraging the students to start earlier and practice a broader selection of questions.
The authors hold a proprietary interest in www.number2.com
Journal
Journal of Educational Computing Research