ACT prep too much, too late, study says

Eleventh-grade students and teachers in the Chicago Public Schools are spending extraordinary amounts of class time preparing for the ACT, but the intense focus on test strategies and item practice is hurting, not helping, performance on this high-skills accountability exam, according to a study released Tuesday by the Consortium on Chicago School Research.

“Students are training for the ACT in a last-minute sprint focused on test practice, when the ACT requires years of hard work developing college-level skills,” said Elaine Allensworth, a co-director at the Consortium and the lead author of the study, “From High School to the Future: ACT Preparation—Too Much, Too Late.” “Students are very motivated to do well on the ACT, so they put a lot of effort into test prep, thinking it will raise their scores. Teachers need to channel this energy into what really matters for the ACT—students doing high-quality work in their courses.”

The research revealed that Chicago teachers commonly spend about one month of instructional time on ACT practice during 11th-grade core classes—driven by pressure to improve scores, despite misgivings about how this practice disrupts lessons. And the time spent prepping is not paying off in higher scores.

ACT scores were slightly lower in schools where 11th-grade teachers reported spending at least 40 percent of their time on test prep, compared to those schools where teachers devoted less than 20 percent of their class time to test prep. This analysis controlled for multiple factors—from a student’s low-income status and incoming test scores, to teacher qualifications and school composition. Thus, comparisons are made between similar schools.

The focus on test prep means students are not making a connection between the work they do in their classes and their ACT scores. About 83 percent of Chicago juniors believe that ACT scores are primarily determined by test-taking skills—a misconception widely shared by their teachers. Only a third of English and science teachers believe the ACT is a good measure of learning in high school, the researchers found.

In fact, high grades in 11th-grade core courses are the strongest predictor of improved ACT scores. Students who started their junior year with a score of 17 (a little above the district’s average) on another standardized test, the PLAN test, and earned a B in their English class that year scored 1 point higher, on average, on the ACT reading test six months later. Students who failed English actually lost ground and scored below 17 on the ACT.

This study relies on qualitative and quantitative data for a cohort of students who were CPS juniors in 2005. This includes test scores from eighth to 11th grades, student transcripts, CCSR surveys and multiple interviews of students and teachers at three Chicago high schools. The report also relied on 2007 survey data reported by CPS juniors and teachers.

Low ACT scores also reflect poor alignment of performance standards from kindergarten through high school and from high school to college. Students appear to be prepared for high school when they enter ninth grade—64 percent of students who took the ACT in 2005 had met the ISAT eighth-grade standards in reading three years earlier. Yet of these students who met state standards, only 30 percent met the ACT reading benchmark three years later. Only those students who exceeded standards in eighth grade were highly likely to meet the ACT reading benchmark; in Chicago, that represents only 855 students.

Moreover, among students still enrolled as juniors, their ninth-grade scores on EXPLORE, another standardized test, suggest Chicago’s incoming freshmen are similar to their peers nationwide. About 42 percent of Chicago freshmen meet the EXPLORE benchmarks in reading, and 41 percent do nationally, but Chicago students lose ground in high school. Only 26 percent of CPS juniors meet the ACT benchmarks for college readiness in reading, while 53 percent of students nationwide hit that reading benchmark score of 21.

The ACT has been administered to all Illinois juniors since 2001, when it became part of the state’s accountability exam. State officials predicted that a test of high-level skills with real-world consequences would be an effective strategy for high school reform. But average ACT composite scores statewide have barely budged over the past five years. State ACT scores are flat—20.3 in 2007 from 20.0 in 2003—while Chicago’s ACT scores inched up to 17.6 last year from 16.8 in 2003.

Requiring the ACT does not lead to more college-oriented instruction because its structure is not an easy one to teach, particularly given the traditional structure of high school courses, according to the report. The ACT is designed to measure college readiness more than learning in particular high school courses. While it incorporates skills taught in high school classes, it is more of a test of thinking and problem-solving skills than a test of specific content knowledge. It is not particularly aligned with any 11th-grade course, although the 11th grade is when most test preparation occurs.

“In the end, moving ACT scores up requires the same strategies as improving graduation rates and better preparing students for college—a focus on the quality of students’ work in their classes, clearly tied to their preparation for the future,” the authors conclude. “There is no quick fix when students lack college-ready skills.” Joining Allensworth in writing the report were Consortium researchers Marcarena Correa and Steve Ponisckiak.