Another COVID Impact: Bad School Data

by Matt Hurt

In a September 14th post, Jim Sherlock referenced some data points that were collected during the pandemic. Specifically he brought up the topics of chronic absenteeism and how the graduation rate didn’t seem to correlate with SOL scores. My intent here is not to refute any specifics; it is to inform readers that there were a variety of aspects that impacted the quality of data that we collected during that time.

First of all, to say that the 2020-21 school year was chaotic is the understatement of the century. Most school divisions began the year in a virtual setting. As the year wore on, students were allowed to come into the school at varying rates. Also during that year, families were ubiquitously allowed to decide whether their students would participate in person, given that was an option.

Many families changed their mind multiple times throughout the year. This by itself caused a great deal of chaos, and it was nearly impossible to accurately reflect each student’s method of instruction during that time period. Try to imagine how this worked out in schools. Johnny’s family chose to have him attend school in person. Then the COVID infection rates in the community increased and Johnny’s family decided that he needed to participate virtually. How hard is it to believe that many kids were marked absent incorrectly when they should have been marked as attending virtually?

Similarly, when students were attending virtually, there was no consistent standard of what was considered attendance across the state for those kids. In some divisions, students were marked present simply if they logged in to their Zoom lessons. That did not mean that they had to participate to be marked as present; they simply had to log in. In other divisions, students were marked absent when they did not turn in their work.

Secondly, when schools were shut down on March 13th of 2020, this put the graduation status of many high school students in jeopardy. Many of these students were underclassmen and were enrolled in SOL courses in which they needed a verified credit to graduate. The Virginia Department of Education granted all students who were enrolled in an SOL course a verified credit for that course if they were passing the course by the time schools closed and they needed that verified credit for graduation requirements. As we know from experience, there are many instances in which grades are inflated relative to SOL outcomes. Given this fact, many students were given the gift of a verified credit who would likely not have earned it that year. Therefore, comparing pass rates in 2022 to graduation rates in that same year is not all that instructive.

In conclusion, collecting data prior to COVID was not an exact science. The data that some divisions submitted to VDOE was more accurate than others. The chaos brought on by COVID significantly decreased the reliability of our data. In other words, the data we collected in 2020 and 2021 is likely not worth the paper on which it is printed.

Matt Hurt is executive director of the Comprehensive Instructional Program.