Scrap the SOLs and Move On

Maybe it’s time for Virginia to scrap the Standards of Learning (SOL) tests.

The SOLs arose in the mid-1990s as a way to provide feedback to the community on how well local schools were performing. It was a worthy experiment. Despite massive increases in spending in preceding years, the quality of education in United States was widely seen as deficient. Backers hoped that transparency would provide teachers, principals, school boards, parents and citizens data they could use to work toward the betterment of their schools.

As with so many reforms enacted with the best intentions, this initiative has gone terribly awry. There is little evidence that SOLs improve anything. Indeed, insofar as the standardized exams encourage teachers to “teach to the test” — more on that in a bit — they may do actual harm.

In short order after their enactment, the SOLs morphed into a means to hold schools “accountable” for poor performance. Schools with low levels of academic achievement were highlighted in local media reports and shamed for failing their students. Newspapers published SOL data for schools within their circulation zones, and parents used the data to guide home purchasing decisions. As parents voted with their feet, affluent households displaced poor households in “good” school districts, and poor households gravitated by default to the “bad” schools. In sum, the tests arguably had the unintended effect of aggravating residential inequality and making it harder for poor schools to improve.

Comparing schools with one another was problematic anyway because educational achievement is strongly correlated with socioeconomic status, the mix of affluent and poor children varied widely by school, and average scores reflected socioeconomic status as much as the quality of the teachers and staff. The Virginia Department of Education (VDOE) possessed the data to adjust scores for socioeconomic status so as to not unfairly penalize low-income schools but, for reasons that remain obscure to me, the department stopped publishing it.

Meanwhile, many teachers, principals and school boards learned how to game the system to dress up scores and avoid the shaming. John Butcher, author of Cranky’s Blog, and I have chronicled numerous scandals around the state — the relaxed test standards, the teacher coaching, the classification of sub-performing students as disabled, and sometimes the outright cheating. Newspaper accounts tend to treat these phenomena as isolated instances that happen to occur in their back yard, but they are in fact commonplace.

Perhaps the most insidious gaming of the system involves teaching to the test. As has been explained to me, the SOLs do not test students’ comprehensive knowledge in a particular subject. Rather, they sample knowledge in sub-topics and assume that if a test-taker gets the answers right for those sub-topics, they will demonstrate the same mastery across the board. Over the years, teachers have learned, to pick an example, that the math SOL will address regular polygons but not irregular polygons, so they spend more time teaching regular polygons and perhaps even skip the irregular polygons. Thus, insofar as teachers teach to the test, meaning that they emphasize certain topics over others, SOLs actually may encourage educational malpractice.

As the emphasis has shifted to holding schools accountable for poor performance, VDOE began using SOL scores to declare schools accredited or unaccredited. (There are various flavors of being unaccredited, depending upon whether schools are deemed to be making progress.) While the state can declare a school “unaccredited,” under the state constitution, schools answer to their school boards. The state does negotiate “Memoranda of Understanding” with chronic laggards but, as Butcher has documented, MOUs consist of educratic mumbo jumbo and are useless in turning schools around. At the end of the day, and not for any lack of trying, accountability remains elusive.

Virginia has doggedly tried to make SOLs work for more than twenty years now. We have enough experience under our belts, I would argue, to draw some broad conclusions. SOLs are deficient as a means of measuring students’ academic achievement; if anything, the teaching-to-the-test phenomenon hurts students. SOLs are useless as a means for improving schools’ academic performance or holding administrators accountable for results; teachers and principals are endlessly creative at gaming the system. And, by influencing people of means to buy houses near “good” schools, SOLs arguably have become an unwitting driver of socioeconomic and racial segregation.

I’m not saying that things will miraculously improve if Virginia did away with the SOLs. They exist for a reason. But we must acknowledge that tweaking and nudging a broken system won’t work either. I don’t have any great suggestions for what we put in place of SOLs. I can say that reform should be bottom-up, not top-down, and it should encourage creativity and experimentation. Failed experiments should be shut down, and successes should be replicated. We cannot afford more business as usual.

There are currently no comments highlighted.

13 responses to “Scrap the SOLs and Move On

  1. It’s not about the SOLS.. that’s a bogus narrative.

    What it IS about is – do you want to MEASURE academic performance and do you want to do it in a Standardized way?

    So .. what’s the answer to that?

    Is the idea of doing away with the SOLs …really a proposal to do away with measuring academic performance and you’d do this because there are ‘problems’ with trying to measure performance… so just do away with the concept itself in Virginia… and everywhere else some kind of measurement is used.. that has ‘problems’ or people using those results to buy houses , etc?

    Come On.. Bacon.. are you really saying the idea of measuring academic performance is not a good idea or are you advocating a “better” way of measuring …different or better than the SOLs?

    How about the system the rest of the world uses … PISA? wanna use that?

    or how about the NAEB –
    National Assessment of Educational Progress (NAEP?

    or just stop measuring because any method will have “problems’?

    this must be Jim Bacon’s version of ‘repeal now” .. and we’ll discuss later what to do … eh?

    you guys kill me. You detail every flaw, pillory every fault.. then call for abandonment.. and not once do you talk about what to fix and how… it’ s just burn and run.

    • I’m not against standardized testing, but I am against testing that creates such powerful incentives to manipulate the results. I think we need to start over.

      • every method of measuring creates the “problems” you are talking about.

        Every other state has some kind of academic performance measurement.

        Every year we read of cheating or other issues like “teaching to the test” on other similar testing… right?

        The SAT requires “teaching to the test” – right?

        heck – if you get a drivers license.. you go learn what’s in that test,

        You cannot get into the military or get a teachers license or get a drivers license without taking a test and have to learn whats going to be tested.

        “Teaching to the test” .. is what we do .. it’s how you get a College Degree!

        If you are actually IN FAVOR of measuring but opposed to the WAY the SOLS measure then I’d love to hear you suggest how to improve it – change it.. to accomplish measurement … better.

        but for you to call for it to be abandoned without advocating something better leaves me with the impression that you’re opposed to measuring academic performance itself..

        surely that’s not what you’re’ advocating. Can you clarify?

  2. How else, other than SOL’s, can you ensure that teachers are teaching what is in the syllabus? The tests can be improved by being comprehensive in the essentials, not testing the fringes. They should not be abandoned. We need quality control. What we should abandon is the notion that all students have identical skills, be they basketball, music, math, or whatever. Even the average scores of selected groups of students need not be identical. That the tests show differences among students, groups of students, schools, teachers, etc., is not a reason for abandoning them.

  3. I agree with Larry and Fred. Before the SOLs, the only thing that was discussed was “more resources.” The SOLs at least forced the system to look at output, results.

    What should happen is that any teacher who is found to have cheated on the SOLs should be suspended or even fired from his/her job. Any administrator who is found to be cheating on the SOLs should not only be fired, but have his/her state credentials revoked.

  4. “Backers hoped that transparency would provide teachers, principles, school boards, parents and citizens data they could use to work toward the betterment of their schools.”


    I hereby revoke Bacon’s Rebellion’s accreditation as a blog.

  5. Here, on this one, I’m with the mob – Sack Bacon, Enforce SOLs with big teeth that bite hard, hurting the crooks and incompetents bad, for the good of kids, their parents and taxpayers.

  6. There’s a lot to this issue but I’m agog that after critics have used the mandated transparency and accountability to pummel the public schools for their “failures” .. “problems”, gaming, cheating, etc.. that the same critics apparently would just abandon the whole concept of measuring academic performance … at all! LORD!

    By the way – VDOE has been working on changing the way that kids are tested. It”s seems to be a big improvement over high-stakes testing in that it’s more frequent assessments that occur in the same classrooms that kids normally sit in – as opposed to going to computer labs. The testing software records the areas where the kid still needs improvement and essentially develops a lesson plan for that kid…

    It would be great if Jim would blog on it.

    but despite the new way of testing.. it still will measure.. academic performance – as it should – with one other huge improvement.. it’s going to be harder to game…. check it out.. get Jim to go talk to VDOE about it.

  7. I am really surprised that anyone with good Libertarian credentials would come out against transparency, against accountability, against treedom of informed choice, against a person’s ultimate right to vote with his feet. The answer to the corruption of public measures of performance is not to stop measuring performance — it’s to fix the measures and eliminate the corruption!

    • I could not agree more. I too am quite concerned about Bacon. For sure he’s now off the reservation. Indeed he might have gone Rogue, over to the other side.

      He can’t even tell us the difference between a principle and a principal! Go figure.

  8. I will add my vote – Nope, Jim, I cannot agree. The problem, if there is a problem, is that the teachers and administrators have such a huge incentive to fudge, finagle and fool with the results. The educational establishment has always hated being measured on its results, and in fairness (returning to a theme with me) schools can only do so much. Parents are the child’s first and most important teacher, and it matters whether the child grows up in an environment that values education. Too many parents (and I do think it is more common now) think they send the kid to school and their job is done. Me, check their homework? That’s your job, teacher!

    And returning to a favorite theme of yours, Jim, too many of those children only have one parent and if working that parent is struggling financially, so even if that parent does want to do more in support they can only do so much.

    But the testing and public release of results does keep some pressure on the schools. The schools can and do make a difference for many challenged children, do offer them the tools to move to a middle-class job and life. The accreditation process does keep a focus on those communities, even if the response from the leadership is mainly finger pointing and the response from the middle class families in those communities is a move out or a move to private school.

  9. I made the following comments on the new post about reforming the SOLs, but it pertains to the comments I first read here, so I thought I would include the comments here, as well.

    I am late to the discussion, but I would like to offer a suggestion that is different than making adjustments at the margin of a system that is not preparing our young people for the future.

    We are discussing Standards of Learning but as with many topics today the name is misleading. We are assessing standards of teaching, or more accurately Standards of Regurgitating. We are measuring students’ ability to remember and regurgitate facts.

    The Chinese educational system, at its highest levels, produces students that are far better at doing this than American students, but are also markedly lacking in critical thinking skills and their ability to create new ways of doing things. They are so indoctrinated with the idea of “one right answer” that they are unable to put together what they know in new ways or to create new insights. This is limiting their progress as a society, just as we are here.

    Probably the students who are the best at regurgitating facts are those who go to medical school. In Med school, they are taught many more “facts”. After residency, they are fully skilled at managing the symptoms of chronic disease while their patients get progressively less healthy. They are taught little to nothing about how to help their patients live full, healthy lives.

    My point is, we can design educational systems that are more efficient at climbing the ladder. But it is very likely that the ladder is leaning against the wrong wall.

    Why are kids so inquisitive in their early years, but either complete dullards or skilled automatons in their later years?

    Robert Kiosaki once commented on those that were smart enough to game the system and pass, and also smart enough not to succumb to it. He said, “As entrepreneurs, the “C” students hire the “A” students, and the “B” students work for the government.

    Education, to me, is a “push” activity. We are trying to push standard methods of behavior and believing into the mass of our population so they can fit a mold and be successful cogs in the machine of our current culture.

    Learning is a “pull” activity that starts with the innate curiosity that is within us all, to a greater or lesser degree, until we stamp it out of most students. Everyone else is just trying to get a job.

    Methods of assessment are useful. But we tend to get what we measure. If we measure conformity, we get conformity, even with under-performers. We need to nurture innovative thinking, confident trial and failure, collaboration, multiple answers, better questions.

    We would be better served to reevaluate what we are trying to achieve and develop the appropriate means of assessing how well we are achieving those ends.

Leave a Reply