To BLOB or Not To BLOB: Large Object Storage in a Database or a Filesystem?
Russell Sears, Catharine Van Ingen, Jim Gray

TL;DR
This paper investigates the optimal storage method for large objects, analyzing fragmentation and performance implications of storing objects in databases versus filesystems based on size and usage patterns.
Contribution
It provides empirical analysis of fragmentation and performance for large object storage, offering guidelines based on object size and access patterns.
Findings
Objects smaller than 256KB are best stored in a database.
Objects larger than 1MB are best stored in a filesystem.
Between 256KB and 1MB, storage choice depends on read/write ratio and overwrite rate.
Abstract
Application designers often face the question of whether to store large objects in a filesystem or in a database. Often this decision is made for application design simplicity. Sometimes, performance measurements are also used. This paper looks at the question of fragmentation - one of the operational issues that can affect the performance and/or manageability of the system as deployed long term. As expected from the common wisdom, objects smaller than 256KB are best stored in a database while objects larger than 1M are best stored in the filesystem. Between 256KB and 1MB, the read:write ratio and rate of object overwrite or replacement are important factors. We used the notion of "storage age" or number of object overwrites as way of normalizing wall clock time. Storage age allows our results or similar such results to be applied across a number of read:write ratios and object…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Data Storage Technologies · Distributed systems and fault tolerance · Parallel Computing and Optimization Techniques
