Reader characteristics linked to detection of pulmonary nodules on radiographs: ROC vs. JAFROC analyses of performance

Akshay Kohli; John W. Robinson; John Ryan; Mark F. McEntee; Patrick C. Brennan

doi:10.1117/12.877958

3 March 2011 Reader characteristics linked to detection of pulmonary nodules on radiographs: ROC vs. JAFROC analyses of performance

Akshay Kohli, John W. Robinson, John Ryan, Mark F. McEntee, Patrick C. Brennan

Author Affiliations +

Proceedings Volume 7966, Medical Imaging 2011: Image Perception, Observer Performance, and Technology Assessment; 79660K (2011) https://doi.org/10.1117/12.877958
Event: SPIE Medical Imaging, 2011, Lake Buena Vista (Orlando), Florida, United States

Abstract

The purpose of this study is to explore whether reader characteristics are linked to heightened levels of diagnostic performance in chest radiology using receiver operating characteristic (ROC) and jackknife free response ROC (JAFROC) methodologies. A set of 40 postero-anterior chest radiographs was developed, of which 20 were abnormal containing one or more simulated nodules, of varying subtlety. Images were independently reviewed by 12 boardcertified radiologists including six chest specialists. The observer performance was measured in terms of ROC and JAFROC scores. For the ROC analysis, readers were asked to rate their degree of suspicion for the presence of nodules by using a confidence rating scale (1-6). JAFROC analysis required the readers to locate and rate as many suspicious areas as they wished using the same scale and resultant data were used to generate Az and FOM scores for ROC and JAFROC analyses respectively. Using Pearson methods, scores of performance were correlated with 7 reader characteristics recorded using a questionnaire. JAFROC analysis showed that improved reader performance was significantly (p≤0.05) linked with chest specialty (p<0.03), hours per week reading chest radiographs (p<0.03) and chest readings per year (p<0.04). ROC analyses demonstrated only one significant relationship, hours per week reading chest radiographs (p<0.02).The results of this study have shown that radiologist's performance in the detection of pulmonary nodules on radiographs is significantly linked to chest specialty, hours reading per week and number of radiographs read per year. Also, JAFROC is a more powerful predictor of performance as compared to ROC.

Citation Download Citation

Akshay Kohli, John W. Robinson, John Ryan, Mark F. McEntee, and Patrick C. Brennan "Reader characteristics linked to detection of pulmonary nodules on radiographs: ROC vs. JAFROC analyses of performance", Proc. SPIE 7966, Medical Imaging 2011: Image Perception, Observer Performance, and Technology Assessment, 79660K (3 March 2011); https://doi.org/10.1117/12.877958

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available