Improving Performance in Java: TOPCAT and STILTS

TL;DR

This paper discusses enhancements in Java applications TOPCAT and STILTS, focusing on multithreading and other strategies to improve performance with large datasets.

Contribution

It introduces new performance optimization techniques, including multithreading, for Java-based data analysis tools, based on recent experience and lessons learned.

Findings

Significant speedups in data processing tasks

Effective multithreading strategies implemented

Enhanced handling of large datasets

Abstract

TOPCAT and STILTS are mature Java desktop applications for working with tabular data that have always had a focus on efficiency for large or very large data sets. This paper presents some progress, experience and lessons learned from efforts over recent years to improve performance further by multithreading key algorithms as well as other strategies.