Digital Collections Explorer: An Open-Source, Multimodal Viewer for Searching Digital Collections

TL;DR

Digital Collections Explorer is an open-source, multimodal web platform that enables natural language and reverse image search over digital collections, making cultural heritage archives more accessible and easier to explore.

Contribution

It introduces a scalable, user-friendly system leveraging CLIP for multimodal search in digital collections, with easy installation and application to diverse cultural heritage data.

Findings

Supports search over hundreds of thousands of images on a standard laptop.

Demonstrates effectiveness across maps, photographs, and PDFs.

Facilitates access to archives with limited metadata.

Abstract

We present Digital Collections Explorer, a web-based, open-source exploratory search platform that leverages CLIP (Contrastive Language-Image Pre-training) for enhanced visual discovery of digital collections. Our Digital Collections Explorer can be installed locally and configured to run on a visual collection of interest on disk in just a few steps. Building upon recent advances in multimodal search techniques, our interface enables natural language queries and reverse image searches over digital collections with visual features. This paper describes the system's architecture, implementation, and application to various cultural heritage collections, demonstrating its potential for democratizing access to digital archives, especially those with impoverished metadata. We present case studies with maps, photographs, and PDFs extracted from web archives in order to demonstrate the flexibility of the Digital Collections Explorer, as well as its ease of use. We demonstrate that the Digital Collections Explorer scales to hundreds of thousands of images on a MacBook Pro with an M4 chip. Lastly, we host a public demo of Digital Collections Explorer.