Research Finds Offender Risk Assessment Tools in U.S. Are Promising, but Questions Remain

The criminal justice system in the United States uses a variety of tools to assess the behavior of criminal offenders, and those risk assessments can have a significant impact on an offender’s fate. A new meta-analysis of the research conducted in the U.S. on these tools shows that – while promising – it is still unclear whether these tools reduce bias against offenders based on race or other factors.

At issue are recidivism risk assessment instruments, or RAIs, which are tools that assess the likelihood that someone who has been arrested for a crime will commit another crime in the future. RAIs are used by professionals in correctional settings, such as parole or probation officers and prison psychologists.

In theory, RAIs reduce bias against minority groups in criminal justice decision-making, and are used to inform where prisoners serve their time and when they are released. RAIs may also be used to inform sentencing and recommendations for intervention or rehabilitation programs. While RAIs are primarily used with convicted offenders, they are sometimes used before an individual has had his or her day in court.

“RAIs are important because they are widely used in the United States in an attempt to reduce biased assessments of recidivism risk and are increasingly being mandated at the local, state and federal level,” says Sarah Desmarais, an associate professor of psychology at NC State and lead author of a paper on the research.

“We know that this sort of structured tool can improve identification of offenders who are more likely to commit crimes in the future,” Desmarais says. “However, a lot of the research has been conducted in other countries and there has not been a comprehensive evaluation of RAIs in the U.S. Which tools are most accurate in identifying recidivism risk in U.S. offenders? And are any of them biased against African Americans or other minority groups? Those are the questions we tried to address in this paper.”

There are, at a conservative estimate, more than 60 recidivism RAIs in use in various parts of the U.S. – most of which are used only in a specific state or municipality. The researchers looked at the 19 most commonly used, which are applied in multiple jurisdictions.

For their meta-analysis, the researchers evaluated all of the research conducted in the U.S. on any of those 19 RAIs. Altogether, the meta-analysis incorporated 53 studies, which looked at a total of 362,323 offenders. Specifically, the researchers looked at how accurate each of the RAIs was at predicting recidivism. The meta-analysis also assessed each RAI’s ease of use and “inter-rater” reliability.” Inter-rater reliability measures how consistent an RAI’s results are among users. So if an RAI is used by two different correctional officers to evaluate a single offender, for example, how likely are those officers to get the same result?

The overall findings suggest that RAIs are pretty accurate.

“But the work also shows that there are a lot of things we still don’t know,” Desmarais says. “There were only a handful of studies for each of the 19 instruments, and very few looked at anything that could give us insight into potential bias against racial or minority groups.”

For example, only two of the studies looked at inter-rater reliability – and both of those studies were evaluating the same RAI. “This is important, because without that inter-rater reliability information, we can’t tell if an RAI is actually increasing consistency – which is why RAIs were developed in the first place,” Desmarais says.

“And we couldn’t evaluate the impact of race on RAI accuracy, because only two of the studies reported on race and accuracy,” Desmarais says. “Those two studies did find almost identical rates of accuracy across races, but you need more than two studies to draw firm conclusions.”

The researchers also found that no single instrument was more effective than the others – it depended on what type of recidivism was being assessed. For example, one tool was the most effective at predicting if offenders would violate the conditions of their parole or probation. But three other tools were more accurate at predicting whether offenders would commit new crimes.

“Our analysis tells us that RAIs can be useful tools in the U.S.,” Desmarais says. “But it also tells us that a lot of additional work needs to be done.”

The paper, “Performance of Recidivism Risk Assessment Instruments in U.S. Correctional Settings,” is published in a special issue of the journal Psychological Services. The paper was co-authored by Kiersten Johnson, a Ph.D. student at NC State; and by Jay Singh of the Global Institute of Forensic Research. The work was funded by the Council of State Governments Justice Center.

-shipman-

Note to Editors: The study abstract follows.

“Performance of Recidivism Risk Assessment Instruments in U.S. Correctional Settings”

Authors: Sarah L. Desmarais and Kiersten L. Johnson, North Carolina State University; Jay P. Singh, Global Institute of Forensic Research

Published: June 6, Psychological Services

DOI: 10.1037/ser0000075

Abstract: With the population of adults under correctional supervision in the United States at an all-time high, psychologists and other professionals working in U.S. correctional agencies face mounting pressures to identify offenders at greater risk of recidivism and to guide treatment and supervision recommendations. Risk assessment instruments are increasingly being used to assist with these tasks; however, relatively little is known regarding the performance of these tools in U.S. correctional settings. In this review, we synthesize the findings of studies examining the predictive validity of assessments completed using instruments designed to predict general recidivism risk, including committing a new crime and violating conditions of probation or parole, among adult offenders in the United States. We searched for studies conducted in the United States and published between January 1970 and December 2012 in peer-reviewed journals, government reports, Master’s theses, and doctoral dissertations using PsycINFO, the U.S. National Criminal Justice Reference Service Abstracts, and Google. We identified 53 studies (72 samples) conducted in U.S. correctional settings examining the predictive validity of 19 risk assessment instruments. The instruments varied widely in the number, type, and content of their items. For most instruments, predictive validity had been examined in one or two studies conducted in the United States that were published during the reference period. Only two studies reported on inter-rater reliability. No instrument emerged as producing the “most” reliable and valid risk assessments. Findings suggest the need for continued evaluation of the performance of instruments used to predict recidivism risk in U.S. correctional agencies.