Technical report: CSVM dictionaries

TL;DR

This paper introduces CSVM dictionaries, an extension of the CSVM format, enabling translation systems between CSVM files with applications demonstrated in chemistry and supported by Python toolkits.

Contribution

It presents the design of CSVM dictionaries, a new extension for CSVM files, along with a working example in chemistry and Python toolkit support.

Findings

CSVM dictionaries enable translation between CSVM files.

Python toolkit fully supports CSVM dictionaries.

Application demonstrated in chemistry context.

Abstract

CSVM (CSV with Metadata) is a simple file format for tabular data. The possible application domain is the same as typical spreadsheets files, but CSVM is well suited for long term storage and the inter-conversion of RAW data. CSVM embeds different levels for data, metadata and annotations in human readable format and flat ASCII files. As a proof of concept, Perl and Python toolkits were designed in order to handle CSVM data and objects in workflows. These parsers can process CSVM files independently of data types, so it is possible to use same data format and parser for a lot of scientific purposes. CSVM-1 is the first version of CSVM specification, an extension of CSVM-1 for implementing a translation system between CSVM files is presented in this paper. The necessary data used to make the translation are also coded in another CSVM file. This particular kind of CSVM is called a CSVM dictionary, it is also readable by the current CSVM parser and it is fully supported by the Python toolkit. This report presents a proposal for CSVM dictionaries, a working example in chemistry, and some elements of Python toolkit usable to handle these files.