Theano-based Large-Scale Visual Recognition with Multiple GPUs

TL;DR

This paper presents a Python-based implementation of AlexNet using Theano, demonstrating effective multi-GPU data parallelism with performance comparable to existing Caffe solutions on fewer GPUs.

Contribution

First open-source Python implementation of AlexNet with multi-GPU data parallelism, enabling accessible large-scale visual recognition training.

Findings

Performance on 2 GPUs matches Caffe on 1 GPU

First open-source Python AlexNet implementation

Demonstrates naive data parallelism effectiveness

Abstract

In this report, we describe a Theano-based AlexNet (Krizhevsky et al., 2012) implementation and its naive data parallelism on multiple GPUs. Our performance on 2 GPUs is comparable with the state-of-art Caffe library (Jia et al., 2014) run on 1 GPU. To the best of our knowledge, this is the first open-source Python-based AlexNet implementation to-date.