Adversity Scores: Great Idea, but Flawed Execution

by James A. Bacon

The College Board, which administers the SAT college entrance exams, caused quite the ruckus recently when it published “adversity” scores for high schools across the country. The idea was to give college admissions officials information to supplement the SATs on the theory that a student who overcomes the challenges of a harsh environment might warrant more consideration than a student with comparable SAT scores who comes from a privileged background. Or, as the Wall Street Journal put it recently, “What if SAT scores could take into account whether a student went to an elite boarding school in New England or a struggling public school in Chicago’s poorest neighborhood?”

I think most people would agree that, all other things being equal, students who surmount the challenges of poverty and family turmoil should be given an edge over students who lived a life of ease and stability.

But the Wall Street Journal went a step further. It hired a Georgetown University data scientist to adjust the average SAT scores for 10,353 high schools where at least 30 students took the exam by their adversity scores. (The adversity scores incorporate 15 factors including the crime rate, poverty levels and other socioeconomic indicators from the student’s high school and neighborhood.) The result was the graph displayed above, which plots median SAT scores against adversity scores. On average, schools with lower adversity scores (more privilege) rank better on SAT scores. But different schools out-perform and under-perform the trend line. Some of the poorest schools punched well above their weight, the WSJ concluded, while some of the wealthiest performed poorly.

I have no problem with a methodology that gives snooty, elitist private schools their comeuppance. Although I attended an “elite” prep school decades ago, I have soured on institutions that have priced themselves beyond the reach of the middle class in order to pay for facilities and programs of little value beyond status signaling. I won’t give them a dime! On the other hand, in the interest of understanding how the world really works, I must take exception to the WSJ’s portrayal of private schools.

The WSJ methodology shows that public magnet schools perform “exceptionally well.” In particular, the Journal highlighted Fairfax County’s Thomas Jefferson High School for Science and Technology (seen above). The school had a low adversity index but its average SAT scores, the second highest average in the country, placed it way above the trend line.

Out of curiosity I checked the performance of two elite college preparatory schools (elite by Richmond standards) with which I am familiar. Both had very low adversity scores — 6 out of 100 for the Collegiate School and 5 out of 100 for St. Catherine’s. Both performed somewhat above the trend line, and both had adversity-adjusted scores just above the adjusted national average. So, they fared respectably under the WSJ methodology but were hardly standouts.

There’s just one problem. As I understand the methodology, the College Board adversity score makes one huge assumption: that the student body of the school shares the demographic and socioeconomic characteristics of the neighborhood in which it is situated. That might be reasonable for public schools, most of which draw students from their immediate neighborhoods. But private schools pull from a much wider market area. More importantly, many private schools provide scholarships to lower-income and inner-city kids who have displayed precisely those characteristics that the adversity scores adjust for.

For example… The Collegiate School provides $3,960,000 in financial assistance to students. Seventeen percent of its 1,681 students qualify for aid. High school tuition runs $27,500. Grants range in size from $1,000 to $27,500; the average grant is $13,580. Located in the West End of Richmond, the Collegiate student body is pretty white — 78%. However, there is a fair number of Asian students (6.6% of the student body, roughly in line with the percentage of Asians in the metropolitan area) and a decent number of African-Americans (7.1%). While some of the black kids came from families of affluent doctors and lawyers a few years ago when my son attended, Collegiate also recruits from poorer parts of town.

The assumption that a private school like Collegiate reflects the demographic make-up of its immediate neighborhood (almost 100% white) is highly misleading. By contrast, the idea of scholarships at a public magnet school like Thomas Jefferson in Fairfax is a non sequitur. It’s a public school. Tuition is free. There are no scholarships. Thomas Jefferson’s admissions criteria are based on academic merit, with the result that its student body is predominantly Asian and, secondarily, white, with a tiny fraction of blacks and Hispanics.

Bacon’s bottom line: If there were a way to measure the adversity of individual students — as opposed to the adversity scores based on neighborhoods — then the College Board would have a really useful tool. Until it does, the methodology perversely punishes private schools that pursue a socially desirable policy of providing financial aid to lower-income students and minorities. Believe me, I carry no water for elite private schools that engage in amenities arms races that drive up the cost of tuition. (My high school now charges an unconscionable $47,500 a year in tuition, although households with incomes up to $200,000 may qualify for assistance. Insane!) But fair’s fair. Give private schools credit for what they do.