Statistical Malpractice: ACT-Tests Edition

by James A. Bacon

Here’s the smiley-face spin on the latest ACT college-readiness scores by Virginia students, straight from the Virginia Department of Education press release:

Virginia students again outperformed their peers nationwide by significant margins this year on the ACT college-admissions test. The performance of Virginia high school graduates improved in all subjects assessed by the four-part test, while the achievement of graduates nationwide declined in English and mathematics, and was flat in reading and science.

Here’s a frowny-face interpretation of the latest data: The percentage of Virginia high school graduates taking the ACT exams declined again in 2020: by two percentage points. Of the 19.3% of grads (not quite one in five) who took the exams, fewer than half (48%) rated college-ready in all four of the English, reading, math, and science benchmarks.

Technically, both versions are accurate. Which spin do you find more believable?

Similar to the more widely used SAT exams, the ACT exams test the college readiness of high school graduates. Virginia has among the highest average test scores of any state in the country. But the Old Dominion has one of the lowest percentage of students taking the exam. Fifteen states require 100% of high school graduates to take the exam. It’s an objective measure of a high school diploma’s worth.

It is reasonable to suggest that the smaller the percentage of students taking the exams, the better the average scores will be. Why? Because college-bound students are more likely to be academically prepared than non-college-bound students. Students who have no intention of going to college have no reason to take the exams, so their lower-than-average scores are not included.

In 2018, 24% of all Virginia grads took the exam, according to ACT data. In 2019, the percentage declined to 21%. In 2020, the percentage declined to 19% — or, according to the more precise VDOE data, 19.3%.

Colleges and universities are becoming increasingly skeptical of the value of standardized test scores, so many college-bound students did not take the tests. However, it is reasonable to assume that no one in Virginia, where testing is voluntary, takes the test unless they aspire to attend college. I find it frightening to think that half the test takers would need remedial work in one or more subject areas.

Bacon’s bottom line: In my admittedly jaundiced analysis, the Northam administration and its progressive allies are presiding over the greatest collapse in K-12 educational achievement in modern Virginia history. Under the circumstances, I would predict that administrators at all levels of the educational bureaucracy to obscure their failure by eliminating standardized tests when possible and spinning the results outrageously when not possible. Bacon’s Rebellion will stay on top of this ongoing story….