Test-set results can predict participants' development in breast-screen cancer detection: An observational cohort study

Basel A Qenam; Tong Li; Abdulaziz Alshabibi; Helen Frazer; Ernest Ekpo; Patrick Brennan

doi:10.1002/hsr2.2161

Test-set results can predict participants' development in breast-screen cancer detection: An observational cohort study

Health Sci Rep. 2024 Jun 17;7(6):e2161. doi: 10.1002/hsr2.2161. eCollection 2024 Jun.

Authors

Basel A Qenam^{1

2}, Tong Li^{1

3

4}, Abdulaziz Alshabibi², Helen Frazer⁵, Ernest Ekpo^{1

6}, Patrick Brennan¹

Affiliations

¹ Medical Image Optimisation and Perception Group, Discipline of Medical Imaging Science, Faculty of Medicine and Health The University of Sydney Camperdown New South Wales Australia.
² Department of Radiological Sciences, College of Applied Medical Sciences King Saud University Riyadh Saudi Arabia.
³ The Daffodil Centre The University of Sydney, A Joint Venture with Cancer Council Sydney New South Wales Australia.
⁴ Sydney School of Public Health, Faculty of Medicine and Health University of Sydney Sydney New South Wales Australia.
⁵ Screening and Assessment Service, St Vincent's BreastScreen Fitzroy Victoria Australia.
⁶ Orange Radiology, Laboratories and Research Centre Calabar Nigeria.

Abstract

Background and aim: Test-sets are standardized assessments used to evaluate reader performance in breast screening. Understanding how test-set results affect real-world performance can help refine their use as a quality improvement tool. The aim of this study is to explore if mammographic test-set results could identify breast-screening readers who improved their cancer detection in association with test-set training.

Methods: Test-set results of 41 participants were linked to their annual cancer detection rate change in two periods oriented around their first test-set participation year. Correlation tests and a multiple linear regression model investigated the relationship between each metric in the test-set results and the change in detection rates. Additionally, participants were divided based on their improvement status between the two periods, and Mann-Whitney U test was used to determine if the subgroups differed in their test-set metrics.

Results: Test-set records indicated multiple significant correlations with the change in breast cancer detection rate: a moderate positive correlation with sensitivity (0.688, p < 0.001), a moderate negative correlation with specificity (-0.528, p < 0.001), and a low to moderate positive correlation with lesion sensitivity (0.469, p = 0.002), and the number of years screen-reading mammograms (0.365, p = 0.02). In addition, the overall regression was statistically significant (F (2,38) = 18.456 p < 0.001), with an R² of 0.493 (adjusted R² = 0.466) based on sensitivity (F = 27.132, p < 0.001) and specificity (F = 9.78, p = 0.003). Subgrouping the cohort based on the change in cancer detection indicated that the improved group is significantly higher in sensitivity (p < 0.001) and lesion sensitivity (p = 0.02) but lower in specificity (p = 0.003).

Conclusion: Sensitivity and specificity are the strongest test-set performance measures to predict the change in breast cancer detection in real-world breast screening settings following test-set participation.

Keywords: breast cancer screening; clinical audit; mammographic test set; radiologist performance.