Optimizing the cutoff for the identification of skin sensitizers by the HaCaSens assay: Introducing an ROC-analysis-based cutoff approach

Abstract

The worldwide restricted use of animal testing makes it challenging to identify the skin sensitizing potentials of newly manufactured products. The HaCaSens assay has shown promise as an in vitro skin sensitizing assay comparable to existing assays, and is currently under pre-validation. However, there is little agreement on how to assess the results of the assay to discriminate sensitizers from non-sensitizers as the stimulation index (SI) cutoff value was arbitrarily chosen without appropriate statistical methods. Here, we investigated the SI cutoff values in identifying sensitizers to obtain the optimal value. Sensitivities and specificities were calculated for a set of 30 test substances, and plotted in receiver operator characteristics (ROC) curves. The SI cutoff values with the highest sum of sensitivity and specificity according to LLNA data were 2.2, 1.8 and 3.0 for interleukin 1α (IL-1α), interleukin 6 (IL-6), and the combination of the two cytokines respectively. Also, the same statistical analysis of human data demonstrated optimal SI cutoff values 2.0, 2.0 and 3.2 for the same respective parameters. When considering the predictive capacity of each possible SI cutoff value determined by ROC curves, the optimal value for HaCaSens is 3.0 for the combination of IL-1α and IL-6 as it had the highest sensitivity (90.9%), specificity (75.0%) and accuracy (86.7%) based on LLNA data. Thus, we recommend the wide use of the SI cutoff value of 3.0 to ensure consistent endpoints.

Keywords: Cutoff; HaCaSens; Interleukin-1α; Interleukin-6; ROC analysis; Skin sensitization.