Analyzing the performance of risk assessment instruments: a response to Vrieze and Grove (2007).

Résumé

In a recent article, Vrieze and Grove (Law Hum Behav, doi: 10.1007/s10979-007-9092-x , 2007) argue that, because of low recidivism base rates and limited predictive accuracy, an actuarial risk assessment instrument (ARAI) may produce decisions about sex offenders that are worse than simply predicting that no one will commit another sex offense. This article examines: (1) the construction and potential overfitting of ARAIs; (2) the meaning, value, and limitations of ROC areas; and (3) the relationship between the operating point that maximizes an ARAI's correct classifications and the legal criterion-"likely to reoffend"-used for sex offender designations. Contrary to what Vrieze and Grove suggest, ARAIs of modest accuracy yield probabilistic information that is more relevant to legal decision-making than just "betting the base rate."