Less is More: Accelerating Faster Neural Networks Straight from JPEG

TL;DR

This paper explores methods to accelerate neural networks processing JPEG compressed images directly from DCT coefficients, reducing computational costs while maintaining accuracy on large-scale datasets.

Contribution

It introduces data-driven strategies for combining DCT inputs and reducing network layers to improve efficiency without sacrificing performance.

Findings

Combining all DCT inputs via learning outperforms manual discarding.

Layer reduction combined with DCT input utilization decreases computational costs.

Approach maintains accuracy on ImageNet dataset.

Abstract

Most image data available are often stored in a compressed format, from which JPEG is the most widespread. To feed this data on a convolutional neural network (CNN), a preliminary decoding process is required to obtain RGB pixels, demanding a high computational load and memory usage. For this reason, the design of CNNs for processing JPEG compressed data has gained attention in recent years. In most existing works, typical CNN architectures are adapted to facilitate the learning with the DCT coefficients rather than RGB pixels. Although they are effective, their architectural changes either raise the computational costs or neglect relevant information from DCT inputs. In this paper, we examine different ways of speeding up CNNs designed for DCT inputs, exploiting learning strategies to reduce the computational complexity by taking full advantage of DCT inputs. Our experiments were conducted on the ImageNet dataset. Results show that learning how to combine all DCT inputs in a data-driven fashion is better than discarding them by hand, and its combination with a reduction of layers has proven to be effective for reducing the computational costs while retaining accuracy.