Development of a Big Data Framework for Connectomic Research

TL;DR

This paper presents a Hadoop-based big data framework with a new C++ library for 3D brain image analysis, enabling scalable processing of connectomic datasets like Kasthuri11.

Contribution

It introduces a novel distributed architecture and a C++ library for 3D image analysis tailored for connectomic research, enhancing scalability and efficiency.

Findings

Successful deployment on Kasthuri11 dataset

Improved scalability for large brain datasets

Effective integration of 3D analysis techniques

Abstract

This paper outlines research and development of a new Hadoop-based architecture for distributed processing and analysis of electron microscopy of brains. We show development of a new C++ library for implementation of 3D image analysis techniques, and deployment in a distributed map/reduce framework. We demonstrate our new framework on a subset of the Kasthuri11 dataset from the Open Connectome Project.