Rationale and objectives: We performed a multipoint rank-order experiment to evaluate variability in observers' sensitivity to small differences in image presentation and to assess observers' performance as a function of the type and number of tasks included.
Methods: Five experienced observers were presented with four sets of chest images that had been compressed at five different levels. Each set contained six images ranging from noncompressed to approximately 60:1-compressed images. Observers were asked to review all images of each case side by side and rank-order the "quality" of each to enable determination of the presence or absence of interstitial disease and/or pneumothoraces.
Results: Observers varied significantly in their ability to detect very small differences among the images (P < 0.001). Those who performed well did so regardless of whether they ranked a specific abnormality in a multidisease or a single-disease setting.
Conclusions: Selected observers can reliably detect very small differences among similar images. These readers could be used to confirm or rule out the need for objective observer-performance-type studies.