Higher SOL Scores: Improved Student Achievement… or Bureaucratic Blarney?

Source: Virginia Department of Education

Source: Virginia Department of Education

by James A. Bacon

Seeming good news from the Virginia Department of Education: Virginia students showed significant improvement in their SOL scores in the 2014-2015 school year. Pass rates increased five percentage points for mathematics (from 74% to 79%) and reading (also 74% to 79%) from the previous year. Pass rates for writing increased two percentage points (from 75% to 77%).

Also encouraging: Hispanic and African-American students closed some of the achievement gap with white and Asian students. “The gap in reading between black and white students has narrowed by two points since more challenging reading tests were introduced,” states the VDOE press release released this morning

“Virginia teachers and students are adapting to the more rigorous standards implemented by the state Board of Education several years ago,” said Superintendent of Public Instruction Steven R. Staples. “The positive trend lines confirm that meeting these new standards is possible, although it will take time for schools to complete the adjustment.”

Bacon’s bottom line: The question is, how real are these gains? The 2014-2015 school  year was the first in which students in grades 3-8 were allowed to retake SOL tests in reading, mathematics, science and history. On average, the performance of students on retakes increased pass rates by about four points on each test. In other words, most of the improved performance reflects a change in administering the tests — a change designed to improve results — not a change in underlying achievement!

Staples defended the change:

By providing a second chance, we get a more complete picture of the performance of schools in preparing students to meet the commonwealth’s high expectations for learning and achievement,” Staples said. “We all understand the limitations of a single, point-in-time test. Some students who initially do not pass may have just had a bad day. And there are students who barely miss the benchmark and just need a little extra instruction in a particular area to achieve proficiency. For these students, an expedited retake offers another opportunity to demonstrate success before the end of the year.

That’s all fine, as far as it goes. But let’s be clear that we’re comparing apples with oranges when comparing 2013-2014 tests with 2014-2015 tests. It’s impossible to say, as VDOE does in its press release, that the results “represent significant progress in the commonwealth’s effort to better prepare students for success in college and careers.” No, it represents the effects of letting failing students re-take the test, which they could not do last year. It is impossible to conclude from this data that performance is actually improving, and to insinuate otherwise is pure political spin.