Good Correctional Policy Starts with Good Data

The City of Richmond is building a new jail to relieve overcrowding. Another way to relieve overcrowding would be to reduce recidivism. But that’s harder than raising steel and concrete, especially when the data doesn’t exist to figure out what works. Photo credit: Richmond Times-Dispatch.

by James A. Bacon

It costs roughly $25,000 a year on average to incarcerate people in jails and prisons across Virginia — totaling more than $1 billion a year for the state Department of Corrections and hundreds of millions more for local governments. One of the few things upon which everyone can agree is that state and local governments could save a big heap of money if they did a better job of reintegrating felons into productive society when they finish serving their time.

A wide variety of government and not-for-profit programs endeavor to help. The problem is, the quality of programs varies widely and it is very difficult to say objectively which ones are effective and deserve more funding, and which ones are worthless and deserve to be terminated.

One program, Kingdom Life Ministries (KLM), which focuses on Richmond City Jail inmates battling substance abuse, had a recidivism rate of only 34%, far lower than the average rate for the jail, concluded Sarah Scarbrough, a Virginia Commonwealth University Ph.D., on the basis of her dissertation research. KLM establishes a relationship with the inmates while in jail, sets concrete goals for reintegrating them into society, and helps them carry out those goals in a strictly regulated halfway-house setting. A key distinguishing feature is the participant’s reliance upon their peers, not just mental health professionals, in supporting them through trying times.

By contrast, Scarbrough told me in an interview, other inmates enter halfway houses that maintain no structure or discipline. “There are as much drugs and prostitution there as on the street!” And in all probability, the recidivism rate at those facilities is much higher.

Virginia is not keeping pace with other states like Oregon, Michigan, Missouri and Tennessee, which have implemented evidence-based practices to drive down recidivism. In Virginia, state programs do not conduct wide-scale evaluations on a regular basis, says Scarbrough. Not-for-profits typically don’t have the resources to do so.

While every program can cite anecdotal stories and offer cherry-picked data, their statistics are often based upon small samples or calculated differently. “These circumstances make it difficult to create a large enough collection of data to deduce well-supported conclusions,” she wrote in the executive summary of her dissertation.

An example of how difficult it is making comparisons: KLM’s recidivism rate of  34% compares to 44% for the Richmond city jails as a whole. The performance gap actually is far bigger than suggested by those numbers, says Scarbrough. The jail’s rate includes only re-arrests in the city. If an inmate were arrested in Henrico or Chesterfield, he wouldn’t be counted as a recidivist.

Also, according to the Virginia Performs website, there are different definitions for recidivism. Some definitions count all offenses, including technical offenses, while others only count when people are convicted of a new crime. Different definitions cover different lengths of time. And there are variations when recidivism tracking begins — when the offender is still in a correctional environment, or after release.

Many, if not most, inmates have problems with substance abuse, and if their drug and alcohol usage is not addressed, they have a greater likelihood of committing a crime. Just as no one is comparing the effectiveness of reentry initiatives, no one is comparing how effectively the state is spending millions of dollars on substance-abuse initiatives, Scarbrough says.

Bottom line: Virginia authorities are wandering in a statistical fog. It is impossible for to efficiently allocate scarce public and philanthropic dollars without good data. Before we spend more money on new jails and prisons, perhaps we should invest in developing a system for tracking the effectiveness of substance-abuse and reentry programs. It would cost more up-front but would pay tangible, measurable dividends within a few years.

Based on her research, Scarbrough argues that KLM, where she now serves on the board, saved the commonwealth $7 million in reduced prison expenditures and nearly $1 million in reduced expenses for emergency medical treatment of inmates over the 3.5-year study period. Extrapolated across all of Virginia’s jails and prisons, she says, the potential exists to save taxpayers hundreds of millions of dollars yearly — and to turn hundreds of inmates into contributing members of society.

9 Responses to Good Correctional Policy Starts with Good Data

  1. re: ” Based on her research, Scarbrough argues that KLM, where she now serves on the board, saved the commonwealth $7 million in reduced prison expenditures and nearly $1 million in reduced expenses for emergency medical treatment of inmates over the 3.5-year study period. ”

    this kind of co-mingling appears ripe for conflict of interest to me.

    I do not know anyone involved but warning bell go off when someone who has or has developed a direct relationship with an organization that was the subject of “research”…

    Even if she was totally apart and separate at the time the research was done, the subsequent association with the organization that was the subject of her original research – makes me queasy.

    Further – numbers count. Small samples relative to the overall numbers is not solid evidence in my view.

    But back before any of this, I question the policies that incarcerate non-violent crime. There is no question that people who are prone to violence against others need to be ridden herd on by the criminal justice system but when we take folks, especially young people, who make serious mistakes in judgement and draw them into the criminal justice system as convicted felons – we damage their chances of getting back to a normal life in society.

    So I question any recidivism stats that do not distinguish the actual reason why someone was incarcerated because I would suspect that the type of crime committed does affect a commitment on the part of the incarcerated individual with respect to getting out of prison and staying out of prison.

    Further, people who are convicted of violent crimes rightly need a lot more attention and monitoring upon release and I would like to see the recidivism rate between violent felons and non-violent felons.

    • LarryG said, “This kind of co-mingling appears ripe for conflict of interest to me. … Even if she was totally apart and separate at the time the research was done, the subsequent association with the organization that was the subject of her original research – makes me queasy.”

      Sarah Scarbrough responds: “My study had 489 subjects, which is much bigger than state wide studies. (Ones using federal data often have more than that, but states are typically under 100). Further, the research was regulated by VCU’s Internal Research Board (IRB) which has very strict guidelines, and also managed by my dissertation committee, so there was strong and strict oversight on all methods and approaches. Also, my relationship with KLM turned into me being on the board after research was concluded. After seeing the effectiveness and the lives changed, I wanted to remain involved and continue to help, but just in a different capacity.”

  2. Actually, Dr. Scarbrough’s “research” thoroughly investigates the relevance of each inmate’s criminal activity to their subsequent recidivism. I would advise taking a closer look at her “research” before making broad sweeping statements. Her research was conducted in order to fulfill the requirements of the doctorate program at Virginia Commonwealth University – not to appeal to the desires of her subjects. So I would again suggest a little more digging prior to insinuating any conflict of interest.

    Furthermore, the”numbers” you are seeking…

    Further – numbers count. Small samples relative to the overall numbers is not solid evidence in my view.

    do not exist.

    Programs such as KLM and others, are not being studied or evaluated on a regular basis. A reality that Dr. Scarbrough’s dissertation is attempting to expose. In fact, she is urging the state to conduct more studies like this one to determine the validity of programs across the Commonwealth. Her findings would show you that your assumptions about non-violent and violent offenders is wildly inaccurate, but we need to evaluate more programs to generate data that can really help the efficiency of our criminal justice system.

    If you are interested in her analysis of the KLM program, and her thorough review of the issue of recidivism, her dissertation is available on her blog – http://sarahscarbrough.com/research/

    • anytime you develop formal ties with the same entity that you are ‘researching’, there is risk of conflicts.

      Even if there are none, it looks not good.

      One paper won’t make or break anyone in the longer run if it turns out to be just an anomaly.

      I don’t have assumptions about violent vs non-violent. I said I wanted to see some recidivism rate comparisons between the two groups.

      But I still think studying one program without comparing and contrasting it with other programs is too small in scale to draw many general conclusions about anything other than THAT program.

      re: the link – Jesus – this reads like an advertisement. good lord!

      what say you Jim Bacon?

  3. I respectfully accept the response but I believe researchers should always stay arms length from the subject of their research lest they give the impression that their objectivity has been compromised.

    I liken this to someone like the AEI doing “research” on Social Security – which they vehemently oppose as a concept. I mean, if they oppose it as a concept, just what am I to think about their “research”?

    Similarly, if the Center for American Progress did a rip snorter of a “research” of all the benefits of SS, I’d be similarly skeptical.

    If Ms. Scarbrough did future research on similar programs and found the to be good or bad, would we wonder if she was using the program she subsequently joined as a standard metric as a basis for her evaluation of the other approaches?

    I think it makes me, at the least, wonder and that’s not a good thing.

    Having said all of this – I sincerely apologize if anything I said did insult or was unfair or unwarranted.

  4. I applaud Ms. Scarborough’s work. It’s something that The Imperial Clown Show in Richmond should be doing for themselves but (as usual) they aren’t.

    However, there is something of a flaw in the economic logic put forth by Ms. Scarborough and Jim Bacon. The economic savings cited in the article assume that the released prisoners no longer create any costs for the Commonwealth. I suspect that is not the case. For many reasons, I’d guess that released prisoners receive considerable assistance from both state and federal programs.

    The magnitude of these programs can be immense. By one estimate, the The amount of money spent on welfare programs equals, when converted to cash payments, about “$168 per day for every household in poverty.

    That’s not only well above the $25,000 per year it costs to incarcerate a prisoner, it is (stunningly) well above the median household income in America.

    http://m.weeklystandard.com/blogs/welfare-spending-equates-168-day-every-household-poverty_665160.html

    Now, this is not “apples to apples”. Individuals are not households and support for the poor requires administrative overhead, etc.

    However, from the referenced article, “… a low-income individual on food stamps may qualify for $25,000 in various forms of welfare support from the federal government on top of his or her existing income and resources—including access to 15 different food assistance programs.”.

    It seems to me that non-recidivism is a start in the economic analysis. The additional category of self-sufficiency would be required before “savings” could be projected.

    Please don’t misunderstand my comments. Non-recidivism is a worthy goal, in an of itself. Since a person returns to jail for committing a crime, non-recidivism means a reduction in the crime rate. It also means that the former prisoner is no longer caged in a miserable jail of prison. However, the state’s job doesn’t end with non-recidivism. That job needs to include an effort before and after release to help the former inmate find work that allows for self-sufficiency.

  5. re: ” That job needs to include an effort before and after release to help the former inmate find work that allows for self-sufficiency.”

    even BEFORE then – putting someone in prison for a non-violent crime, in a prison with offenders convicted of violence makes the release process much harder.

    People convicted of non-violent crimes need to serve their just punishment but putting them with violent felons is worse than a just punishment and worse than that, as DJ points out, at the other end of the process – few convicted felons ever make it back to a productive enough person to be a net tax payer rather than entitlement consumer.

    We set a pretty low bar when the goal is to not go back to prison – anyhow, in my view.

    When you get a chance – for giggles and grins, you should go to your local police arrest blotter and look for 2 things:

    1. – the persons age
    2. – their crime – relative to whether it is violent or not.

    Our criminal justice system grinds up young people who used bad judgement on something like drugs – and it spits them out as convicted felons whose remaining lives are truly purgatory.

    We have MORE people in PRISON as a PERCENT of our POPULATION than any other nation on earth – even some of the totalitarian regimes.

    think about that – not only in terms of how many we have in prison – but what happens when they get out and need food, shelter, medical care, etc. Bacon quoted 25K a year to keep them in prison.
    I wonder what the average sum of entitlements is for released folks?

  6. As a Sociologist, I have had the opportunity to both read the methodology section of Scarbrough’s dissertation and to listen to her speeches. As a volunteer with Richmond City Jail, I am also very familiar with the structure of the Jail and the diligent input of RCJ staff that goes way beyond Scarbrough’s interaction. Professionally, I am absolutely confident in my analysis that her methodology is EXTREMELY flawed. It was not a mixed method, it was a scattered method that used a plethora of methodological designs that do not work well together. Her sampling design is non existent. Simply comparing two populations is NOT sampling, which is required for establishing a Control and Experimental group within the Sample and more importantly required to establish the methodological validity of a study and the ability to assert that a certain intervention (IV) is responsible for changes in the DV. This is regression not correlation. 2) She did not use a true “Experimental Design” because she did not sample and control for extraneous factors within the total populations studied. Therefore, there was no control group and any comparison between the two would be methodological invalid. 3) Case studies are very rigorous in their requirements, none of which were met by Scarbrough’s study. Simply conducting interviews or conducting passive observation of groups/sessions does not constitute a “case study.” 4) Surveys are NEVER qualitative. 5) She speaks about the use of secondary data. Secondary data analysis is a method within itself. It CANNOT be soundly used in collaboration with an Experimental Design. IF Scarbrough was attempting to establish that the KLM Model was solely responsible, or even statistically or methodologically significantly responsible for variations in the recidivism rate, whe would have to have isolated her sample first and then controlled for very concrete extraneous, mediating/intervening variables that could effect the relationship between the IV and the DV. SHE DID NOT! As a Ph.D., I am surprised that the Faculty Advisors at VCU would have overlooked such glaring flaws. I am also shocked that the community and the media has allowed such flawed research to become a catalyst. And while I do not believe that Scarbrough deliberately intended to do such flawed research, if one is versed in methodology, I would certainly hope that she would responsibly re-evaluate it, and become the expert that her ungrounded status has allowed her to be, by actually re-doing her work. There are GREAT implications to/for research. It poses the potential to truly create positive change and valuable programs. On the converse flawed research in and on which programs and policies are based also pose the contradictory implication of wasting money on programs that are ineffective because they were not based on plausible understandings of relationships and TRUE values. Dr. Zoe Spencer

  7. PS….Therefore, to the article. Good Correctional Policy Starts With Good Data….Well….This is NOT IT.

Leave a Reply