by James C. Sherlock
This article is the first in a series about COVID-associated learning losses in Virginia public schools.
The contribution I hope to make is to measure learning losses and correlating factors in each of 132 school divisions horizontally against its own pre-COVID learning assessment results.
That is different than comparing Richmond to Falls Church to Wise County vertically. We will do that too, but only in knowledge differentials — gains and losses — across all grade levels, not in specific levels of knowledge attained by the students before and after COVID interruptions.
Then we will seek correlation of learning losses with other factors. At this level of aggregation of statistics, correlation is what can be done. Causation assessment requires far more information than is available to the public.
I have left out race as a factor on purpose, at least at this time. I have found that when race is included all of the rest of the data tend to be ignored. A mistake in my view. I have included a factor of percentage of students in each school division economically disadvantaged for this data run. I may check it against racial correlations later.
The measures of student achievement used here for measuring learning losses are:
- the last SOLs taken before COVID in 2018-2019 and
- the SOLs in the post-COVID-shutdown year of 2021-22.
Resources over that period were teachers, kids and their parents. Some turnover in teachers and kids, but not significant at this level of aggregation. The kids were three years older, replaced by younger ones in each grade. Since SOL testing does not begin until 3rd grade, virtually all that took SOLs in 2021-22 were in the system in 2018-19.
You will see that some divisions — teachers, students and their parents together — navigated the three years between the spring of 2019 and the Spring of 22 well. Some very well. Others failed in what they tried to achieve. Some badly.
As I roll out the data in a series of articles I think readers will find the learning loss data and its horizontal and vertical correlations informative.
And in some cases surprising.
Since 2018-19, much education data have been either missing or unusable because of COVID interruptions. I decided to fill in that gap in a way that I hope will provide meaningful lessons and fair assessments of correlation of learning losses.
Horizontal data fields — Fredericksburg example. To show what types of correlation are coming, I offer one example to show the scope of the project. Here is a look at the entire horizontal spreadsheet element so far for Fredericksburg Public Schools. I have the same for each of Virginia’s districts.
You will note that Fredericksburg’s 2020-21 chronic absentee statistics correlate with massive declines in 2021-22 SOLs from already poor 2018-19 SOL scores in Fredericksburg. VDOE is still attempting to confirm the 71% number.
But we can see the two data elements correlate: poor attendance and bad learning losses.
The learning losses also correlate with late resumption of in-person learning in 2020-21. We see that the division had a relatively standard number of economically disadvantaged kids and students with disabilities compared to state averages, and a high percentage of English Learners against the state average.
We also see that Fredericksburg has adopted the Virginia Tiered Systems of Supports (VTSS) framework designed to improve academics, behavior, and social-emotional wellness. Most school divisions have not. This approach did not seem to help Fredericksburg schools mitigate COVID learning losses.
Vertical comparisons. But it is not only internal division correlation that I am seeking. We will look at vertical correlations (and lack of correlation) across all school divisions in later articles in this series.
You will see that some of the data fields are pending. VDOE has agreed to provide me data for 2018-19 chronic absenteeism so that we can see trends in that domain. I may add data fields to the master later.
I will roll this out a little at a time.
This first article shows vertical results — best and worst. Correlations will come later.
- the divisions with best learning gains and smallest learning losses– those SOLs revealing less than 3% declines in student pass rates of SOLs between Spring of 2019 and Spring of 2022 are here.
- the worst of the learning losses — those SOLs revealing more than 20% declines in student pass rates of SOLs between Spring of 2019 and Spring of 2022 — are here.
Reading gains and losses. A case can be made that reading skills can be maintained by individual students above grade 3 without school support. Reading gains and losses across the 132 divisions are not shown but are listed below.
- Gains in reading skills over the three years were seen in Lexington, Buena Vista, Clarke County, Falls Church, Alleghany County, Colonial Beach and Botetourt County.
- The worst losses in reading skills were (between 15 and 19% lower pass rates in 2021-22) in Hopewell, Charles City County, Bath County and Franklin.
Best school divisions in mitigating learning losses. So what do we see among the best performing school divisions in preventing learning losses? Color coding goes from lime green (outstanding) to red (bad) and every shade in between so that viewers can see visually what went on.
- The size of the student population does not correlate to learning loss.
- Students in these schools (and across Virginia) suffered their smallest learning loss in the subject that they were most likely to do on their own — reading.
- You will also see that only one school division in the state was able to maintain student SOL scores in math — Lexington City.
- Lexington City Public Schools, a small division, started with good scores and ran the table raising the performance of their students. Students actually improved their scores over the COVID years. Norton and Staunton raised the bar or prevented significant learning loss on three SOLs.
- Some of the rest of the schools started with high scores and maintained them. Some with poor or bad scores maintained them.
- Staunton’s 16 point increase in English Writing score was amazing. Similarly Norton’s 13 point increase and Highland County’s 11 point boost in the same SOL. From before COVID.
But we also see in a different view that Lexington above all, Highland County, Middlesex, Norton and Staunton students maintained skills in one or more subjects in addition to reading and writing:
- math (Lexington only among 132 school divisions),
- history and social science; and
- science.
If we take out reading, for which some case can be made could have been maintained by the students and parents themselves at home more than the other subjects, we see that writing skill loss mitigation dominated.
Worst school divisions in mitigating learning losses. How about the worst performing divisions. The attached spreadsheet shows only the SOLs in which more than 20% fewer students passed in the Spring of 2022 than passed the same test in the same division in 2019.
What are some of the things we see?
- Franklin, Harrisonburg and Hopewell were complete train wrecks. Petersburg went from very bad to inexplicable.
- Some of the divisions that finished 2019 with excellent results suffered major learning losses. Galax, Amelia County and Caroline County are examples. Some went from bad to worse.
- Major learning losses in mathematics were found in some surprising divisions. Surprising to me anyway. Alexandria, Chesterfield County, Galax, Henrico County, Madison County, Orange County, Stafford County, Suffolk. There are more, but I did not expect those.
- There were no reading losses of that magnitude, though writing skills suffered major losses in many.
- Major SOL losses in science were nearly constant, followed by History and Social Science, the only subjects that students learn almost exclusively at school.
The school division with major learning learning losses in three or more SOLs were: Franklin and Harrisonburg with four and with three were Amelia County, Amherst County, Brunswick County, Caroline County, Galax, Hopewell, King and Queen County, Madison County, Newport News, Northampton County, Nottaway County, Portsmouth, Radford and Suffolk.
I choose my words carefully here. Whatever they did over those three years failed.
That does not assign blame, and we will look at other correlations, but they failed to do what they tried to do.
Much more follows.
