Efficient labeling of retinal fundus photographs using deep active learning

Samantha K. Paul; Ian Pan; Warren M. Sobol

doi:10.1117/1.JMI.9.6.064001

16 November 2022 Efficient labeling of retinal fundus photographs using deep active learning

Samantha K. Paul, Ian Pan, Warren M. Sobol

Author Affiliations +

Journal of Medical Imaging, Vol. 9, Issue 6, 064001 (November 2022). https://doi.org/10.1117/1.JMI.9.6.064001

Abstract

Purpose

To compare the performance of four deep active learning (DAL) approaches to optimize label efficiency for training diabetic retinopathy (DR) classification deep learning models.

Approach

88,702 color retinal fundus photographs from 44,351 patients with DR grades from the publicly available EyePACS dataset were used. Four DAL approaches [entropy sampling (ES), Bayesian active learning by disagreement (BALD), core set, and adversarial active learning (ADV)] were compared to conventional naive random sampling. Models were compared at various dataset sizes using Cohen’s kappa (CK) and area under the receiver operating characteristic curve on an internal test set of 10,000 images. An independent test set of 3662 fundus photographs was used to assess generalizability.

Results

On the internal test set, 3 out of 4 DAL methods resulted in statistically significant performance improvements (p < 1 × 10^{− 4}) compared to random sampling for multiclass classification, with the largest observed differences in CK ranging from 0.051 for BALD to 0.053 for ES. Improvements in multiclass classification generalized to the independent test set, with the largest differences in CK ranging from 0.126 to 0.135. However, no statistically significant improvements were seen for binary classification. Similar performance was seen across DAL methods, except ADV, which performed similarly to random sampling.

Conclusions

Uncertainty-based and feature descriptor-based deep active learning methods outperformed random sampling on both the internal and independent test sets at multiclass classification. However, binary classification performance remained similar across random sampling and active learning methods.

Citation Download Citation

Samantha K. Paul, Ian Pan, and Warren M. Sobol "Efficient labeling of retinal fundus photographs using deep active learning," Journal of Medical Imaging 9(6), 064001 (16 November 2022). https://doi.org/10.1117/1.JMI.9.6.064001

Received: 7 May 2022; Accepted: 31 October 2022; Published: 16 November 2022

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $24.00

Non-members: $28.00 ADD TO CART

JOURNAL ARTICLE
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Data modeling

Photography

Binary data

Receivers

Performance modeling

Eye models

Ophthalmology

Purpose

Approach

Results

Conclusions

Show All Keywords

Keywords/Phrases

Search In:

Publication Years