Design and Evaluation of Interactive Proofreading Tools for Connectomics

• Published in: IEEE transactions on visualization and computer graphics Vol. 20; no. 12; pp. 2466 - 2475
• Main Authors: Haehn, Daniel, Knowles-Barley, Seymour, Roberts, Mike, Beyer, Johanna, Kasthuri, Narayanan, Lichtman, Jeff W., Pfister, Hanspeter
• Format: Journal Article
• Language: English
• Published: United States IEEE 31.12.2014; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adult; Algorithms; Animals; Computer Graphics; Connectome - methods; Data visualization; Editing; Female; Humans; Image segmentation; Imaging, Three-Dimensional - methods; Interactive systems; Internet; Male; Mice; Rendering (computer graphics); Software; Three-dimensional displays; Young Adult; Segmentation; Quantitative Evaluation; Proofreading; Connectomics
• ISSN: 1077-2626; 1941-0506
• DOI: 10.1109/TVCG.2014.2346371

Proofreading refers to the manual correction of automatic segmentations of image data. In connectomics, electron microscopy data is acquired at nanometer-scale resolution and results in very large image volumes of brain tissue that require fully automatic segmentation algorithms to identify cell boundaries. However, these algorithms require hundreds of corrections per cubic micron of tissue. Even though this task is time consuming, it is fairly easy for humans to perform corrections through splitting, merging, and adjusting segments during proofreading. In this paper we present the design and implementation of Mojo, a fully-featured single-user desktop application for proofreading, and Dojo, a multi-user web-based application for collaborative proofreading. We evaluate the accuracy and speed of Mojo, Dojo, and Raveler, a proofreading tool from Janelia Farm, through a quantitative user study. We designed a between-subjects experiment and asked non-experts to proofread neurons in a publicly available connectomics dataset. Our results show a significant improvement of corrections using web-based Dojo, when given the same amount of time. In addition, all participants using Dojo reported better usability. We discuss our findings and provide an analysis of requirements for designing visual proofreading software.