Retention study looks at public school test scores

When Chicago Public School students scored higher this year on reading and math tests than they had previously, some commentators suggested that the scores were inflated by students who had been held back under a new policy that prohibits social promotions.

Public school students who are struggling academically are now held back instead of being automatically promoted, and critics suggest test scores are higher system-wide because the retained students, who would have scored poorly at a higher grade level, are retaking the test at a lower grade level and getting better scores.

Not so, says a research team at the School of Social Service Administration and the Consortium on Chicago School Research led by Melissa Roderick, Associate Professor in SSA, and John Easton, Deputy Director of the consortium. The researchers recently released an analysis showing that students who are held back are actually holding down the scores instead of raising them.

The data brief, “Adjusting Citywide Iowa Tests of Basic Skills Scores for Student Retention in Grades Three, Six and Eight,” finds that when the scores of retained students in the third, sixth and eighth grades are removed, the 1998 test scores are higher than reported in both reading and math. In fourth and seventh grades, when the retained students’ scores are included as if they had taken the tests in those grades, the scores are lower than reported. But even after these adjustments, the overall system-wide scores are still up from 1997.

The 1998 reading scores for the public school system remain about the same after being adjusted for the effects of retained students.

Easton said that this is the type of puzzle the SSA/Consortium research group was designed to resolve. “We follow the Chicago Public Schools very closely, and one of our areas of expertise is test score analysis,” said Easton. “When this speculation about test scores started floating around, we figured that this was a perfect question for us to answer.”

The test-score question is only one piece of a three-year study being conducted by Roderick on the Chicago Public Schools’ new promotion and retention policy. In 1997, the Chicago Public Schools board decided to end social promotions. Now in order to advance to the next grade, students in the third, sixth, eighth and 10th grades have to meet a set of standards, including minimum achievement scores on the Iowa tests and classroom exams.

“We’re looking at case studies in schools and talking with a large number of students, parents and teachers so we can study the long-term effects of this policy,” said Easton. “There are multiple questions with many pieces: Does holding students back provide them an incentive to do better in school? If they are required to attend summer school, does that reverse summer learning loss and give them skills in the fall? And what happens over time to the kids who have to repeat grades?”

Easton said he and Roderick plan to release analyses regularly over the next three years. “We want to do the new policy justice,” he said. “This is not a thumbs up or thumbs down kind of thing.”