The Fix Was In: Use and Abuse of Survey Data

by James A. Bacon

The Barnes & Thornburg final report into racism at the Virginia Military Institute has done its job of generating loads of negative headlines about the military academy. “Run by White men, for White men,” seems to be the most quotable quote. Predictably, there is no evidence that a single reporter read past the executive summary, which, as I explained yesterday, was a politicized, agenda-driven distillation of the extensive research conducted by investigators who, though not without their biases, painted a complex and nuanced picture of race relations.

The B&T summary conclusion that “racial and gender disparities exist” is based entirely upon the perceptions of a handful of Black VMI cadets. The report cites no documentary evidence of racism on the part of the VMI administration. The problem is alleged to be rooted in “the culture.” Accordingly, the perception of Black cadets, as gleaned in personal interviews and a lengthy anonymous survey, form the basis of B&T’s conclusions.

Here’s what the B&T summary doesn’t tell you: Of the 540 survey responses from cadets, only twelve came from African Americans. (That fact appears only in the appendix.) For most questions, the African-American responses split down the middle — six agreed (strongly or somewhat) with statements supportive of the racism allegations while six disagreed (strongly or somewhat).

Thus, when the B&T executive summary makes statements like this — “according to survey results of current cadets, half of African American cadets strongly or somewhat agree that there is a culture of racial intolerance at VMI” — the finding was based on the responses of six cadets who felt that way — six of 102 African American cadets (2021 enrollment).

Similarly, of the 73 cadets interviewed by investigators, only 17 were African American.

The perceptions of White cadets, alumni, faculty and staff differed markedly from those of their Black counterparts, as did the perceptions of Hispanics and Asians (although the differences were less pronounced). The executive summary dismisses their views as of no consequence. Indeed, the perceptions of Whites are cited as evidence of their failure to recognize the existence of pervasive racism. (The summary does not opine on the similar failure of Hispanics and Asians to recognize racism.) Remarkably, the executive summary gives no credence to the views of the roughly half of African American respondents who disagree with the proposition of racial intolerance.

No consideration is given in the executive summary to the possibility that Black cadets’ perceptions of racism in an era of Black Lives Matter were influenced by their political leanings (a survey question that was asked). The report presents zero evidence that racial problems such as the “not uncommon” use of racial pejoratives is any more common at VMI than any other higher-ed institution. Perhaps most egregiously, the executive summary never explores the possibility that the adversity-driven Rat Line might tear down racial differences and create fraternal bonds between individuals of different races in a way that no other Virginia higher-ed institution replicates.

Bacon’s Rebellion has argued that the conclusions of investigation were foreordained. Governor Ralph Northam prejudged the outcome when he wrote in a to VMI’s Board of Visitors of “a clear and appalling culture of ongoing racism.” The Governor’s Office, including its chief diversity officer, was intimately involved in drafting the Request for Proposal outlining the goals and objectives of the investigation, and members of the Governor’s Office dominated the process by which the winning bidder, Barnes & Thornburg, was selected. The law firm had highlighted its commitment to combating racial injustice on its website and in its appeals to the Northam administration.

Perhaps most damning is the fact that the executive summary systematically excluded reference to questions that yielded answers less supportive of the predetermined findings.

  • When asked if VMI cadets socialize and “hang out” in racially integrated groups, 58% of African American cadets agreed and only 34% disagreed.
  • When asked if VMI faculty care about getting the views and perspectives of all types of cadets, 75% of African American cadets and only 16% disagreed.
  • When asked if “I feel part of the community at VMI,” 67% of Black cadets agreed and 33% disagreed.
  • When asked to agree or disagree with the proposition that news media reports of racially intolerant conduct at VMI do NOT accurately reflect the culture, 58% of African American cadets agreed and 42% disagreed.
  • When asked if VMI’s method of addressing reports of racially intolerant behavior is appropriate, 50% of African American cadets agreed and 34% disagreed.

But the bias in the executive summary runs deeper than its use and abuse of survey data. I will address other issues in future posts.