New study questions validity of ACT exam

ACT test

A new study from the National Bureau of Economic Research (NBER), entitled "Improving College Performance and Retention the Easy Way: Unpacking the ACT Exam," questions the validity of the ACT as a predictor of a student's future performance in college.

The paper was written by Dr. Devin G. Pope, assistant professor at the University of Chicago's Booth School of Business; Dr. Eric P. Bettinger, associate professor at Stanford University's School of Education; and Brent J. Evans, a doctoral student in higher education at Stanford University.

"Not surprisingly and confirming a long line of literature, we find a strong correlation between higher ACT composite scores and positive college outcomes," the authors write. "However, this overall correlation masks an important pattern: Mathematics and English scores are much more tightly correlated with college success than are Reading and Science scores. In fact, after controlling for Mathematics and English scores, Reading and Science provide essentially no predictive power regarding college outcomes."

As a result, the authors say, focusing solely on the English and Mathematics test scores would greatly improve the ACT exam's predictive validity.

"The authors note that because colleges get the score breakdowns and the composite scores, there is nothing to prevent admissions officers from considering only some parts of the ACT, or even of weighting the different parts of the test in different ways," writes Inside Higher Ed's Scott Jaschik. "But they found that, overwhelmingly, colleges fail to do so and instead rely on a composite score that the authors find anything but reliable."

"An email statement from Jon Erickson, ACT interim president of the education division says the ACT is an achievement-based test that is used for multiple goals and purposes beyond just admissions or predicting overall student success, such as college GPA or retention," writes Education Week's Caralee Adams. "He says that all four subject areas are important in college and the composite score represents the best overall college readiness predictor."

The primary dataset for the study, containing information on all students who matriculated to a four-year public university in Ohio in 1999, was provided by the Ohio Board of Regents.