Managing Complex Structured Data In a Fast Evolving Environment

TL;DR

This paper introduces a domain-specific language in Common Lisp designed to unify and manage complex, evolving criminal data formats across jurisdictions, facilitating interoperability and adaptability in law enforcement systems.

Contribution

The paper presents a novel Lisp-based language that effectively specifies and handles complex, changing criminal data structures, enabling scalable and flexible data management.

Findings

System supports over 50 users across the U.S.

Enables easy handling of complex evolving data

Facilitates interoperability across jurisdictions

Abstract

Criminal data comes in a variety of formats, mandated by state, federal, and international standards. Specifying the data in a unified fashion is necessary for any system that intends to integrate with state, federal, and international law enforcement agencies. However, the contents, format, and structure of the data is highly inconsistent across jurisdictions, and each datum requires different ways of being printed, transmitted, and displayed. The goal was to design a system that is unified in its approach to specify data, and is amenable to future "unknown unknowns". We have developed a domain-specific language in Common Lisp which allows the specification of complex data with evolving formats and structure, and is inter-operable with the Common Lisp language. The resultant system has enabled the easy handling of complex evolving information in the general criminal data environment and has made it possible to manage and extend the system in a high-paced market. The language has allowed the principal product of Secure Outcomes Inc. to enjoy success with over 50 users throughout the United States.