Evidence-Based Social Policy

by James A. Bacon

In the 1980s, two University of Pennsylvania criminologists published a research study on the impact of mandatory arrests in cases of domestic violence in Minneapolis, Minn. They found that the arrests led to lower rates of domestic violence. Based on that one study, police departments across the country began instituting mandatory arrests. But experiments conducted in other cities found mixed results. In Omaha and Charlotte, violence increased.

Therein lies a cautionary tale. The logic is slowly sinking in among legislators that it’s a good idea to base public policy on the evidence. It’s good to ask the question, “Does this policy do what it’s supposed to do?” But not all systems for measuring results are created equal.

A new paper by David D. Muhlhausen with the Heritage Foundation argues that Congress should invest a bit more not only to measure results of its programs but to measure them according to the precepts of methodologically sound social science. His logic applies equally to the General Assembly.

Mulhausen advocates the “gold standard” of evaluation designs: Setting up control groups, randomizing the assignation of individuals to those groups, and conducting experiments in multiple locations. To do otherwise, is to risk reporting ambiguous results and nullify the whole purpose of conducting the experiment. He recommends the following: (1) New or reauthorized programs should specifically mandate experimental evaluation of the program; (2) the experimental evaluations should be large-scale, multi-site studies; (3) legislators  should specify the types of outcome measures to be used to assess effectiveness.

To do otherwise is to base public policy on anecdotal evidence — typically in a political context in which self-interested parties are usually the ones generating the anecdotes. Testimony orchestrated by special interests simply cannot be trusted to provide an accurate depiction of reality.

Share this article


(comments below)


(comments below)


2 responses to “Evidence-Based Social Policy”

  1. What? A test plan?

    Success and Failure Criteria?
    Features to be tested and not to be tested?
    When to Suspend a test or Resume it?
    How the test will be performed?
    What will be the deliverable items from the test?

    Gee golly. You think if legislators had to agree on these items as part of the analysis of their policies that A) there would be a lot less wrangling about whether the policy works B) a lot less useless policy?

    Politics consists of a lot of undocumented finger pointing: “It works”, “It doesn’t work”. You think if we had to agree on how to decide, we would have a better shot at designing a policy that does work?

  2. Then there is the case of Gould Arkansas and Quartzite Arisona. Gould banned any organization from meeting without permission from the council, and Quartzite declared martial law.

    Both in response to citizens groups who tried to hold them accountable.

Leave a Reply