Pre-COVID Test Results Show a Failed Public Education System

by James C. Sherlock

I have questions in my own mind about the quality of Virginia public schools.

In search of answers I invested several weeks full time in building into a spreadsheet what I consider some of the critical metrics among both Virginia public schools in general and ten different school districts that I chose.  

For each of those districts I recorded data on: 

  • demographic groups by racial cohort, economically disadvantaged, and English learners;
  • school investment; 
  • chronic absenteeism; 
  • SOL reading and math performance of each demographic group in each district; and  
  • Compared them to state averages in each metric.

I chose and paired the ten different school districts (of 133) in an attempt to get a cross section of urban, suburban and rural districts in Northern Virginia, the Richmond area, Southwest Virginia, Hampton Roads and Southern Virginia.

I used the 2018-19 school year, the last year before COVID, to provide a baseline for learning losses and what those schools need to do going forward.  

The data reveal enormous problems with the basic building blocks of education. 

Performances by Virginia students statewide on standardized math and reading tests in 2018-19 were objectively terrible. The only way to read those results is that many of the public schools and some entire school districts are broken. 

The Board of Education writes in its preamble and vision statement that its primary focus is equity:

“The mission of the Board of Education and Superintendent of Public Instruction, in cooperation with their partners, is to develop policies and provide leadership that improve student achievement and prepare students to succeed in postsecondary education and the workplace, and to become engaged and enlightened citizens.”

Equal opportunity is very important, but it needs to be attained in a successful school environment, not one in which a large number of kids cannot read or multiply.

Virginia’s SOL reading and math pass rates, confirmed by NAEP exams in the same year, show that many public schools have failed. Utterly.  

Before COVID learning losses.

School Systems Chosen for evaluation. Note the pairings I chose to get representation from urban, suburban and rural districts with pairs from Northern Virginia, Hampton Roads, Richmond area, Southern Virginia and Southwest Virginia.

  • Loudoun County
  • Fairfax County
  • Chesapeake 
  • Norfolk
  • Richmond City
  • Henrico County
  • Roanoke County 
  • Wise County
  • Martinsville
  • Henry County

Span of the data. I didn’t use all of the data available, but this is more than enough to draw rough conclusions for further investigation.

Pay attention to not just the values relative to the state and to other districts which are color-coded for easy visual reference. 

Pay attention to the absolute values, which are on the whole horrible, both in the SOL and NAEP test results.

I have provided data on state and school district demographics, school spending, chronic absenteeism broken out by racial cohorts and the disadvantaged and English learners cohort. Aligned with that information I have recorded Math and Reading SOL pass rates for each of the same cohorts in the state and each district.  

SOL. On the SOL a scaled score of 400 or higher of a possible 600 is considered passing. Those are the rates that you see. All items on SOL tests have been reviewed at least twice by committees of Virginia classroom teachers. Only items approved by these teachers as fair and aligned with the commonwealth’s subject and course content standards appear on SOL tests.

Color coding. Finally, I color coded the data fields to show visually how each district compares with the state averages in each field. The fields in dark green indicate the best performances among the 9 school districts; the fields in dark red the worst. The ones in between have appropriate colors in those spectra.

Relative SOL Performance of School Districts 2018-19. As you scan the spreadsheet, you will see some things you likely did not expect.  Or at least I did not. When looking at SOL pass rates across the demographic, economic and English learner cohorts, the school systems in my sample of 10 finished in the following relative order:

The Best. Glancing across the color-coded SOL relative performances for all of the demographics in each school system, the three top-performing school systems in both math and reading of the ten I chose were:

  • Wise County – easily the best
  • Chesapeake
  • Roanoke County

The Worst. 

Again, glancing across SOL results for all demographics, the worst performing school districts were:

  • Richmond – uniquely bad
  • Norfolk – poor, but does well in reading with Hispanics and English learners
  • Henrico County – poor, but does well with white kids

Key Factors 2018-19

I think every data point I recorded is important, but there are several that seem from the data to be somewhat determinant in reading and math performance.  

Ability to pay, spending per pupil, and percentage of school budget for instruction

Interestingly, spending per pupil does not show up as a controlling factor. Loudoun County spent $15,740 per student and had a generally good performance relative to the state and above average performances by Blacks and Asians but had bad results from their Hispanic and disadvantaged kids.

Wise County spent $9,684 per student and nearly every one of its demographic cohorts outperformed their counterparts in Loudoun.

Money spent per child generally aligns with ability to pay. The outliers there were Henrico, which ranked 42nd among 133 districts in ability to pay but spent $3,500 a year or 25% less per child than Richmond, which ranked 33rd in ability to pay.  

The other outlier was Martinsville, ranked 136th in ability to pay, spent $11,629 annually compared to surrounding Henry County, which ranked 133rd and spent almost $2,000 less per child. Much of that difference can be explained by the fact that Henry County spent 68% of its school budget on instruction compared to Martinsville’s 60%.

Henrico County spent the highest percentage of its school budget on instruction at 72.1%. State average was 67.5%.

Location. Urban systems chosen performed worse than suburban or rural systems;

Chronic absenteeism. Absenteeism mattered except in the case of Wise County, which posted the best SOL performances and some of the worst chronic absenteeism.  

Chesapeake turned in the best performance on chronic absenteeism. Not coincidentally, it had by far the most aggressive program in the state for referring kids and their parents to Juvenile and Domestic Relations courts for excessive absences.  

The districts with the lowest rates of chronic absenteeism among disadvantaged kids were Chesapeake and Roanoke County. Those with the highest absenteeism among that same cohort were Wise County and Richmond.  

Economically Disadvantaged. Percentages of disadvantaged kids, who generally underperform, matter, both to the teaching and the results. The success story was Wise County, with nearly 60% disadvantaged (state average 40%) and the highest SOL pass rates. 

English Language Learners. Percentages of English learners matter very significantly in overall school district SOL performance. Of course they underperform significantly. Statewide teacher shortages in that specialty make filling the teacher slots a big issue  It  adds another concern to the crisis at the border.  

Fairfax County had over 55,000 English learners composing nearly 30% of the student body, by far the highest percentage of any of the ten districts. I suspect that Fairfax County also teaches kids with more different native languages than any of the rest. Loudoun had 16% English learners.  

Wise County had only 20 English learners, 3 tenths of 1% of the student population. The lack of English learners seems to account for much of the district SOL performances overall but doesn’t explain the outperformance by its White kids and the massive outperformance by its Black kids in math. 

In fact, Wise County’s outperformance across the board in math was truly remarkable. None of the other ten counties was close. While Wise had only 20 English learners, it also had only 18 Asians.

Hispanics. The best results for Hispanics in reading and math were recorded in Wise County, Chesapeake, Martinsville and Roanoke County. Among those, only Martinsville has a significant percentage of English learners at 9%.

Asians. Asian kids significantly outperform every other demographic in reading and math and drive up overall SOL scores in Loudoun, with almost 25% Asians, Fairfax County with almost 20% and Henrico with 12.5%. Those were the only districts among my ten with more than 4% Asian population.  

Competing factors. Wise County has almost no Asians, yet still outperformed Loudoun, Fairfax and Henrico. That may indicate that English learners have a bigger impact to the down side than Asians have to the upside of district SOL averages.

Surprisingly, at least to me, Hispanic and disadvantaged kids underperformed state averages in math and reading in both Loudoun and Fairfax Counties. That correlates with those counties’ outsized percentages of English learners that are likely both Hispanic and disadvantaged.

Statewide National Assessment of Educational Progress (NAEP) testing 2018-19

The NAEP test is given periodically to specific grades in schools nationwide.  That program provides each state its grades but does not break them down any further. Tests in math and English reading were given to Virginia 4th graders in the 2018-19 school year.

The attached spreadsheet contains a segment that provides those 2018-19 NAEP scores.

Those tests tell us a lot about the academic performance of our schools and also provide a reference point against which to measure the efficacy of SOLs in those subjects given to Virginia 4th graders in the same year, which I will show in the next installment.  

The NAEP tests appear to be tougher than the SOLs. Our kids tested very poorly on them. Again, I am talking in absolute rather than relative (to other states) terms.  

Two examples — the rest can be seen in the spreadsheet:

  • Only 48% of Virginia 4th graders tested proficient or better in math, 39% proficient or better in reading.   
  • 49% of Virginia’s Black 4th graders tested below basicilliterate — in reading, only 19% proficient or better. Fourth grade is, as we know, the first year in which kids are expected to start to read to learn.  
  • I’ll say it again. Half of Virginia’s Black 4th graders could not read.

That is utterly unacceptable and ruinous for those children. And the VDOE thinks its most important mission is equity, not basic math and reading skills.  

It is impossible to consider those results as outliers. There were too many children tested. They did not have learning to lose in COVID.  

And most have now finished the 6th grade. Retention in grade rates are very low.

Source of Data / Add your favorite school district

Your local school district may not be the exception that you may think it is  If I did not choose yours, you can add it to the spreadsheet to see how it compares.  

The source of the data was VDOE School Quality Profiles both the State Report and Browse by Division.  

Having selected either the state or a school division, Click Enrollment for those data, Assessments for SOL scores, and ESSA for chronic absenteeism.  

  • In Enrollment, click on show data to get fall membership by subgroup and use 2018-19 data to align with SOLs.  
  • In Assessments use 2018-19 SOL data.  Remember these are pass rates only.  Click on each cohort.
  • In ESSA, click on View 2019 ESSA School Quality Indicator … and scroll down to Chronic Absenteeism.  These are also 2018-19 data.

Next – Individual schools

This has been a very useful exercise for me. I learned a lot. The results could launch dozens of Ph.D and Ed.D theses.

Next time we will look at performances down to the individual school level in Chesapeake and Loudoun.  

I should have that finished in a couple of days. It is at least differently informative than this one.  

It will make the case for school choice — both charters and open enrollment — like nothing you may have ever seen.

Share this article


(comments below)


(comments below)


36 responses to “Pre-COVID Test Results Show a Failed Public Education System”

  1. vicnicholls Avatar

    Hey Jim, I’m going to look at this, and see if I can’t add VB (one of the school systems where the majority of the board hates me) and maybe Portsmouth. I’ve gotten the ear of one of the elected ones in Portsmouth and want to see how they react. I want to start pushing them harder and they know it.

    1. John Harvie Avatar
      John Harvie

      Sorry to see VA Beach board has gone all female. Think Dan Edwards will be missed.

      Kempsville schools USED to be good when I my stepchildren attended them years ago. Doubtful now.

      1. vicnicholls Avatar

        Dan probably want retirement, he aint no spring chicken any more.

    2. James C. Sherlock Avatar
      James C. Sherlock

      Let me know if you have any difficulty and I’ll add them.

      1. vicnicholls Avatar

        You are the bomb!! I am going to get some of these people on the record about where they are at, and they can then justify spending my tax dollars on stuff that doesn’t educate.

  2. James Wyatt Whitehead Avatar
    James Wyatt Whitehead

    Fantastic work Captain! What in the world is Martinsville spending 40% of the school budget on? 40% non instructional spending seems too high.
    Richmond is spending 35% of the budget on non instruction. The schools spending under 30% on non instruction appear to do better.

    1. James C. Sherlock Avatar
      James C. Sherlock

      It is possible to find the Martinsville school budget online. Statewide the averages are 67.5% on instruction and 32.5% on non-instruction. So yes, Martinsville is pretty far out of line.

    2. James C. Sherlock Avatar
      James C. Sherlock

      What we need are school boards that think in absolute terms about whether the kids are being educated, not how they compare to the rest of the state.

      As I wrote, in absolute terms our school system is broken.

      1. James Wyatt Whitehead Avatar
        James Wyatt Whitehead

        Those many hours you spent on this is worth it! Revealing in so many ways. It should be forwarded to school boards and superintendents. Wise County. You really have to want to go there, one of the most isolated places in Virginia and they have set a high bar that is the envy of wealthy and populous school districts in Northern Virginia. Roanoke too seems to have distilled a winning formula. A demanding school board or a highly skilled superintendent?
        My experience tells me it is often the leadership of one driven public servant that can build a coalition and part the waters of adversity.

        Smaller districts under good leadership seem to thrive despite that lack of investment in the school budgets. It leads me to wonder if massive districts such as Loudoun, Fairfax, and Prince William would be better served by partitioning the district into smaller ones.

    3. Dick Hall-Sizemore Avatar
      Dick Hall-Sizemore

      A lot of this discrepancy is attributable to economies of scale. The smaller school divisions, such as Martinsville, have certain fixed administrative expenses that end up taking up a larger percentage of their overall spending. For example, the non-instructional costs constituted 45.6 percent of the expenditures for Bath County schools; 41.8 percent for Colonial Beach; 38.3 percent for Buena Vista, 36.8 percent for West Point; and 34.3 for Highland County.

  3. If your goal is creating racial “equity” the place you start is ensuring that 4th graders know how to do basic reading and math. If they fall behind in those skills, they will never keep pace with the kids who can read and calculate. All the diversity-equity-inclusion rhetoric is an elaborate excuse for the failure of Virginia’s educrats.

    1. Nancy Naive Avatar
      Nancy Naive

      Simple solution. Stop wasting their time teaching the biographical history of failed despots and generals with their wars.

      Spend 6 hours on math. There is no CRT with math. Once you get past the 1800s, the old white guys fade away. In fact, old everything fades away. If you ain’t done it by 25, good chance you won’t.

      I was in my 60s before I heard of Greenwood, OK. I was in my 40s before I’d heard of Rose-whatever in Florida. I’m guessing there was a whole lotta bad $#!t done to Black Americans I still haven’t heard, nor did my daughter in her K-12 less than 15 years ago.

      Might as well teach that Southern trees bear strange fruit. What is it youse guys always say about “not knowing history”?

      1. Matt Adams Avatar
        Matt Adams

        Perhaps the problem was you were educated in the US for some of your formative years.

    2. Nancy Naive Avatar
      Nancy Naive

      Never heard of this in my Virginia K-12… now that I think of it, Tuskegee wasn’t mentioned in either way.

      Colfax, La., 1873: This was a direct attack on Black men getting the right to vote during Reconstruction. After Whites contested the result of the 1872 election, Black men and a mostly Black state militia holed up around the parish courthouse to protect the local government. On Easter Sunday, April 13, 1873, they were surrounded by a White mob that set the courthouse on fire and shot anyone who emerged. It is estimated that 62 to 81 African Americans were killed.

      Elaine, Ark., 1919: There were dozens of racist attacks and massacres across the country in the Red Summer of 1919. One of the worst was in Elaine, Ark., in which at least 200 Black farmers and their families were slaughtered. The farmers had recently unionized and were planning to bypass the unfair sharecropping system.

      1. James Wyatt Whitehead Avatar
        James Wyatt Whitehead

        About 40% of the content in US HISTORY an 11th grader will learn will be about the history of minorities. Not bad for 20% of the population.

        1. Nancy Naive Avatar
          Nancy Naive

          20%? Try 45%, nearing the 50% mark. How much of that US history involves the SW States?

      2. James C. Sherlock Avatar
        James C. Sherlock

        The subject is why kids are not learning to read and calculate 20 years into the 21st century in Virginia. Do you have a contribution to make?

        1. Nancy Naive Avatar
          Nancy Naive

          Yeah. The written word is the new Latin. Video and sound are the language of tomorrow. Teach them who and what we really were –racists — and sew a TI-84 into their forearm.

          Hey! A good place to start — Adult Education. Teach the half of our country, who doesn’t know, what the difference between a self-guided tour and a violent insurrection is.

          1. James C. Sherlock Avatar
            James C. Sherlock

            “Teach them who and what we really were — racists.” A little self reflection going on there Spanky?

          2. Nancy Naive Avatar
            Nancy Naive

            Uh yep, and so are you. The difference? I ain’t proud of it.

            BTW, This isn’t aimed at Biden… it’s aimed at the 6.

            You must be happy…


  4. Nancy Naive Avatar
    Nancy Naive

    2021– the Conservative solution to end discrimination. Stop teaching about discrimination… and supress the Black vote instead.

    1. DJRippert Avatar

      You can teach about discrimination all day long if the kids also learn reading, writing and arithmetic. Like most right leaning citizens I think the history curriculum is pretty bad and needs revision. However, fixing that curriculum won’t fix the lack of learning in math and English.

  5. Matt Hurt Avatar
    Matt Hurt

    Please be aware that the chronic absenteeism data is the data point in this analysis that is probably the least valid. Think about how this data is collected- taking roll in the classroom each morning. There’s a lot of stuff going on, and in some places accuracy is stressed more than others.

    1. James C. Sherlock Avatar
      James C. Sherlock

      Got it. Have you got any insight into why the Wise County SOL pass rates were so high?

      1. Matt Hurt Avatar
        Matt Hurt

        Yes sir, and I’ll try to answer that question as succinctly as possible.

        We became more reliant on our data. Analysis of that data demonstrated that there were some teachers who had more significantly at-risk students who outperformed teachers who had fewer at-risk students. When we talked to those very successful teachers of very at-risk students, we found they consistently 1) made sure that they taught explicitly and exactly what the standards required students to know, do, and understand, 2) had very positive relationships with students and their parents/guardians, and 3) had higher expectations of their students (and themselves) than did their peers.
        Since the data demonstrated that our at-risk students could be successful, this informed the expectations of administrators in the schools and the central office. A significant portion of the evaluation of SOL teachers and principals was based on SOL results. Kindergarten through 2nd grade teachers’ (who don’t have SOL tests) evaluations were highly aligned to student outcome measures such as PALS, division assessments, and other measures aligned to the Standards of Learning. The accepted truth in the division was that it was not up to the students or the parents/guardians to ensure students were successful, rather that success was the sole responsibility of the adults employed by the division. Basically, everyone lined up and pulled in the same direction towards better student outcomes as measured by the SOL test.
        There was a significant focus on student outcomes throughout the year, not just at SOL testing time. We worked with teachers to develop interim benchmarks to monitor student progress. Expectations for student success became part of the de facto culture and data was discussed and used consistently. Teachers in many grade levels had 90 minutes of planning, and to support instruction, the school board allowed half that time to be spent daily for remediation. That board act was not necessary, because most teachers pulled students during their planning for additional instruction as needed, evidenced by their formative assessments.
        Teachers were given the autonomy to meet the needs of their students to ensure their success in any way they saw fit, so long as it was legally, ethically, and morally correct. The teachers understood the expectations, and the administrators allowed them to meet those expectations in the manner that worked best for them. For example, the division ceased textbook adoptions, and instead allocated textbook funds to schools on a per pupil basis so that teachers could purchase materials that they felt would do the trick.

        I don’t think this is an all-inclusive list, but I do think that this covers the major points.

        1. James C. Sherlock Avatar
          James C. Sherlock

          Thank you, I will refer to this often.

  6. Nancy Naive Avatar
    Nancy Naive

    Hmmm, I think we’re seeing this in the vaccine-hesitant Red States already…

  7. Dick Hall-Sizemore Avatar
    Dick Hall-Sizemore

    Putting this comparison together involved a lot of time and work and is a useful way to analyze the results of school divisions. I commend you.

    There are two additional measures that could shed some more light on the jobs school divisions are doing: dropout rate and pupil/teacher ratio.

    As you pointed out, DOE has this data on its website. One can examine the results by school division and by individual schools. However, that website does not contain a tool that would enable one to easily compare divisions, such as you have done.

    Obviously, the DOE staff have the capability of comparing the achievement by division, using measures of race, educational advantaged, etc. One would hope that they have done so.

    With the information that DOE has, it could easily identify those schools that are doing really well, as well as those that are doing poorly. It could then take whatever it is that the well-performing schools are doing and apply it in those divisions that are not doing well.

    Doing that would take strong leadership at the state level. Because the Virginia Constitution vests supervision of schools with the local school boards, the state traditionally has been loath to tell local school boards how to run their schools, except for high-level policies such as the Standards of Quality, accreditation standards, and teacher certification requirements.

    1. James C. Sherlock Avatar
      James C. Sherlock

      All of the in-school crime data are available as well, but I chose not to include it.

      1. vicnicholls Avatar

        Now THAT would have made for interesting reading. LOL.

        1. Matt Hurt Avatar
          Matt Hurt

          That discipline data is not that consistent. It is reported differently in different schools and divisions. In some places all discipline referrals are reported, and others, only egregious violations are documented.

          1. James C. Sherlock Avatar
            James C. Sherlock

            The data show that to be true. Unless the offense is a felony, some principals clearly won’t refer kids to law enforcement. Others will if the kid doesn’t respond to anything else and has hurt or threatened to hurt someone. I will show that when I publish an assessment and comprehensive data for each school in Loudoun County.

    2. James C. Sherlock Avatar
      James C. Sherlock

      And good data analysis shop could build a program that would automatically generate these data for every school system in the state in this or your suggested expanded format. They have the data. They just have to display it in a color-coded heat map as I have done.

      When you see a similar design formatted to display each school, using Loudoun as a guinea pig, you will see how much more actionable that format would be for superintendents and school boards. I will post that one soon. Again, the data is already formatted at VDOE. All they need is a program to generate the spreadsheet for each division and school in each division.

      Frankly, the individual school spreadsheet is the best thing they could do for the school divisions.

  8. DJRippert Avatar

    “Fairfax County had over 55,000 English learners composing nearly 30% of the student body, by far the highest percentage of any of the ten districts. I suspect that Fairfax County also teaches kids with more different native languages than any of the rest.”

    And Fairfax has gone from being one of the top school systems in America to a mediocre system within the state.

    There is a cost to being a sanctuary county.

Leave a Reply