720 words; 3 minutes to read
Audio summary by volunteer Averi Brailey.
This post draws largely on the work of Dr Jennifer Kamorowski and colleagues.
The criminal justice system is making increasing use of ‘risk assessment’ instruments and procedures to inform decisions about bail, sentencing, parole and probation. The idea is that whether a person is released should depend in part on the risk that the person might commit another crime.
Predicting future actions by individuals is generally itself a very risky activity, with a high likelihood that our predictions turn out to be wrong. This is true in every field, not just in criminal justice. Our predictions about how people in a group may do x or y are reasonably accurate, but our predictions about specific people are generally quite inaccurate. For example we know with a fair degree of accuracy roughly how many murders will occur in a year in Canada, but we have very little idea of who will actually be involved.
Predictions about individual future actions are also subject to many biases. Many studies have shown that juries, judges, prosecutors and parole officers also have biases that affect their perceptions of risk and even of guilt.
Mistakes carry serious consequences
When the decisions involve a person’s liberty, as is the case with bail or parole, a high possibility of errors must be a very serious concern. It is important to take steps to prevent future crimes, but if we are too cautious in using risk assessment, as is often the case, the result is that many people will be and are imprisoned or otherwise constrained without just cause, and that is a serious violation of basic human rights.
One way the justice system has tried to overcome this problem is through the use of structured risk assessment instruments (SRAI). These instruments usually involve a series of questions or measures that, taken together, are considered to have a higher accuracy in making predictions about individual’s future actions. The instruments are supposed to be used by people who are specially trained, including to minimize their personal biases.
These instruments, now widely used in criminal justice in many countries including Canada. For example all federal prisoners in Canada are evaluated for parole against such an instrument.
How well do risk assessment instruments work?
A considerable amount of research has examined the degree to which these instruments are effective. The consensus of the research is that SRAIs are generally more accurate than unaided predictions made by experts (such as psychologists). That’s the good news.
But that good news has to be qualified by 3 provisos. First, the instruments all still have relatively high error rates. To take a hypothetical example, an instrument that predicted accurately 85% of the time whether a person would commit another crime would be considered very, very accurate. But that would still mean that 15% of the time, people would be unfairly penalized by the continued denial of their liberty. And some people would still be released who would commit another crime. That’s a large number of wrong predictions with very serious negative consequences.
Second, the final decisions about risk are not made by trained experts but by judges or supervisors in the bail and parole system – people who often have no training and may have little understanding of the mathematics of risk as outlined here. In other cases, such as civil commitment hearings in the US, juries must interpret risk assessment data, which they often struggle to do fairly.
Third, as shown by Kamorowski and colleagues, even the experts using these instruments are often inclined to override the results and substitute their own personal judgments even though those judgments are much more likely to be biased on grounds such as ethnic background. As they report, The current objectivity and outcome research… suggests that when an evaluator uses a risk assessment tool, the results do not necessarily reflect an objective evaluation.
Caution always needed in judging risk
The reality is that human decisions will always have some degree of error, and some of that error will in turn be due to bias. These tools can help but they will never totally overcome the problems. That is why Risk assessment tools should be used to identify ways to manage and reduce risk – but their vulnerability to evaluator bias indicates they should not be used to justify significant deprivations of freedom.