Full Content is available to subscribers

Subscribe/Learn More  >
Proceedings Article

Characterizing and optimizing rater performance for internet-based collaborative labeling

[+] Author Affiliations
Joshua A. Stein, Andrew J. Asman

Vanderbilt Univ. (USA)

Bennett A. Landman

Vanderbilt Univ. (USA) and Johns Hopkins Univ. (USA)

Proc. SPIE 7966, Medical Imaging 2011: Image Perception, Observer Performance, and Technology Assessment, 79660M (March 03, 2011); doi:10.1117/12.878412
Text Size: A A A
From Conference Volume 7966

  • Medical Imaging 2011: Image Perception, Observer Performance, and Technology Assessment
  • David J. Manning; Craig K. Abbey
  • Lake Buena Vista, Florida | February 12, 2011

abstract

Labeling structures on medical images is crucial in determining clinically relevant correlations with morphometric and volumetric features. For the exploration of new structures and new imaging modalities, validated automated methods do not yet exist, and so researchers must rely on manually drawn landmarks. Voxel-by-voxel labeling can be extremely resource intensive, so large-scale studies are problematic. Recently, statistical approaches and software have been proposed to enable Internet-based collaborative labeling of medical images. While numerous labeling software tools have been created, the use of these packages as high-throughput labeling systems has yet to become entirely viable given training requirements. Herein, we explore two modifications to a typical mouse-based labeling system: (1) a platform independent overlay for recognition of mouse gestures and (2) an inexpensive touch-screen tracking device for nonmouse input. Through this study we characterize rater reliability in point, line, curve, and region placement. For the mouse input, we find a placement accuracy of 2.48±5.29 pixels (point), 0.630±1.81 pixels (curve), 1.234±6.99 pixels (line), and 0.058±0.027 (1 - Jaccard Index for region). The gesture software increased labeling speed by 27% overall and accuracy by approximately 30-50% on point and line tracing tasks, but the touch screen module lead to slower and more error prone labeling on all tasks, likely due to relatively poor sensitivity. In summary, the mouse gesture integration layer runs as a seamless operating system overlay and could potentially benefit any labeling software; yet, the inexpensive touch screen system requires improved usability optimization and calibration before it can provide an efficient labeling system.

© (2011) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation

Joshua A. Stein ; Andrew J. Asman and Bennett A. Landman
"Characterizing and optimizing rater performance for internet-based collaborative labeling", Proc. SPIE 7966, Medical Imaging 2011: Image Perception, Observer Performance, and Technology Assessment, 79660M (March 03, 2011); doi:10.1117/12.878412; http://dx.doi.org/10.1117/12.878412


Access This Proceeding
Sign in or Create a personal account to Buy this proceeding ($15 for members, $18 for non-members).

Figures

Tables

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

Advertisement

 

 

 

  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Proceeding
Sign in or Create a personal account to Buy this proceeding ($15 for members, $18 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.