Reducing Storage in Large-Scale Photo Sharing Services using Recompression

TL;DR

This paper introduces novel recompression codecs, ROMP and L-ROMP, that significantly reduce storage requirements for large-scale photo sharing services while maintaining quality and compatibility, leading to substantial storage and bandwidth savings.

Contribution

The paper presents two new photo recompression codecs, ROMP and L-ROMP, that improve storage efficiency and are compatible with JPEG, enhancing large-scale photo storage management.

Findings

ROMP achieves 15% better compression than standard JPEG.

L-ROMP provides 28% lossy compression with imperceptible quality loss.

Storage reduction of 0.3-0.9x and bandwidth savings of up to 31%.

Abstract

The popularity of photo sharing services has increased dramatically in recent years. Increases in users, quantity of photos, and quality/resolution of photos combined with the user expectation that photos are reliably stored indefinitely creates a growing burden on the storage backend of these services. We identify a new opportunity for storage savings with application-specific compression for photo sharing services: photo recompression. We explore new photo storage management techniques that are fast so they do not adversely affect photo download latency, are complementary to existing distributed erasure coding techniques, can efficiently be converted to the standard JPEG user devices expect, and significantly increase compression. We implement our photo recompression techniques in two novel codecs, ROMP and L-ROMP. ROMP is a lossless JPEG recompression codec that compresses typical photos 15% over standard JPEG. L-ROMP is a lossy JPEG recompression codec that distorts photos in a perceptually un-noticeable way and typically achieves 28% compression over standard JPEG. We estimate the benefits of our approach on Facebook's photo stack and find that our approaches can reduce the photo storage by 0.3-0.9x the logical size of the stored photos, and offer additional, collateral benefits to the photo caching stack, including 5-11% fewer requests to the backend storage, 15-31% reduction in wide-area bandwidth, and 16% reduction in external bandwidth.