Caffe con Troll: Shallow Ideas to Speed Up Deep Learning

TL;DR

This paper introduces Caffe con Troll, an optimized version of Caffe that significantly improves CNN training throughput on CPUs, enabling efficient hybrid CPU-GPU training by leveraging batching optimizations.

Contribution

We developed Caffe con Troll with internal modifications to enhance performance, demonstrating substantial throughput gains and enabling efficient hybrid CPU-GPU CNN training.

Findings

4.5x throughput improvement over Caffe on popular networks

End-to-end training time proportional to CPU FLOPS

Efficient hybrid CPU-GPU CNN training enabled

Abstract

We present Caffe con Troll (CcT), a fully compatible end-to-end version of the popular framework Caffe with rebuilt internals. We built CcT to examine the performance characteristics of training and deploying general-purpose convolutional neural networks across different hardware architectures. We find that, by employing standard batching optimizations for CPU training, we achieve a 4.5x throughput improvement over Caffe on popular networks like CaffeNet. Moreover, with these improvements, the end-to-end training time for CNNs is directly proportional to the FLOPS delivered by the CPU, which enables us to efficiently train hybrid CPU-GPU systems for CNNs.