Toward an Automated HPC Pipeline for Processing Large Scale Electron Microscopy Data

We present a fully modular and scalable software pipeline for processing electron microscope (EM) images of brain slices into 3D visualization of individual neurons and demonstrate an end-to-end segmentation of a large EM volume using a supercomputer. Our pipeline scales multiple packages used by th...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Vescovi, Rafael, Hanyu Li, Kinnison, Jeffery, Keceli, Murat, Salim, Misha, Narayanan Kasthuri, Uram, Thomas D, Ferrier, Nicola
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 06.11.2020
Subjects
Image segmentation
Supercomputers
Workflow
Workstations
Online AccessGet full text

Cover

More Information
Summary:We present a fully modular and scalable software pipeline for processing electron microscope (EM) images of brain slices into 3D visualization of individual neurons and demonstrate an end-to-end segmentation of a large EM volume using a supercomputer. Our pipeline scales multiple packages used by the EM community with minimal changes to the original source codes. We tested each step of the pipeline individually, on a workstation, a cluster, and a supercomputer. Furthermore, we can compose workflows from these operations using a Balsam database that can be triggered during the data acquisition or with the use of different front ends and control the granularity of the pipeline execution. We describe the implementation of our pipeline and modifications required to integrate and scale up existing codes. The modular nature of our environment enables diverse research groups to contribute to the pipeline without disrupting the workflow, i.e. new individual codes can be easily integrated for each step on the pipeline.
ISSN:2331-8422