Biomedical article retrieval using multimodal features and image annotations in region-based CBIR

Daekeun You; Sameer Antani; Dina Demner-Fushman; Md Mahmudur Rahman; Venu Govindaraju; George R. Thoma

doi:10.1117/12.838973

18 January 2010 Biomedical article retrieval using multimodal features and image annotations in region-based CBIR

Daekeun You, Sameer Antani, Dina Demner-Fushman, Md Mahmudur Rahman, Venu Govindaraju, George R. Thoma

Proceedings Volume 7534, Document Recognition and Retrieval XVII; 75340V (2010) https://doi.org/10.1117/12.838973
Event: IS&T/SPIE Electronic Imaging, 2010, San Jose, California, United States

Abstract

Biomedical images are invaluable in establishing diagnosis, acquiring technical skills, and implementing best practices in many areas of medicine. At present, images needed for instructional purposes or in support of clinical decisions appear in specialized databases and in biomedical articles, and are often not easily accessible to retrieval tools. Our goal is to automatically annotate images extracted from scientific publications with respect to their usefulness for clinical decision support and instructional purposes, and project the annotations onto images stored in databases by linking images through content-based image similarity. Authors often use text labels and pointers overlaid on figures and illustrations in the articles to highlight regions of interest (ROI). These annotations are then referenced in the caption text or figure citations in the article text. In previous research we have developed two methods (a heuristic and dynamic time warping-based methods) for localizing and recognizing such pointers on biomedical images. In this work, we add robustness to our previous efforts by using a machine learning based approach to localizing and recognizing the pointers. Identifying these can assist in extracting relevant image content at regions within the image that are likely to be highly relevant to the discussion in the article text. Image regions can then be annotated using biomedical concepts from extracted snippets of text pertaining to images in scientific biomedical articles that are identified using National Library of Medicine's Unified Medical Language System® (UMLS) Metathesaurus. The resulting regional annotation and extracted image content are then used as indices for biomedical article retrieval using the multimodal features and region-based content-based image retrieval (CBIR) techniques. The hypothesis that such an approach would improve biomedical document retrieval is validated through experiments on an expert-marked biomedical article dataset.

Citation Download Citation

Daekeun You, Sameer Antani, Dina Demner-Fushman, Md Mahmudur Rahman, Venu Govindaraju, and George R. Thoma "Biomedical article retrieval using multimodal features and image annotations in region-based CBIR", Proc. SPIE 7534, Document Recognition and Retrieval XVII, 75340V (18 January 2010); https://doi.org/10.1117/12.838973

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available