How good are MatLab, Octave and Scilab for Computational Modelling?

TL;DR

This study evaluates the accuracy and stability of MatLab, Octave, and Scilab across different operating systems using standard and NIST data sets, focusing on numerical computations and statistical functions.

Contribution

It provides a comprehensive accuracy assessment of three popular computational platforms across multiple operating systems using standard benchmarks and NIST data.

Findings

MatLab generally produces more accurate results than Octave and Scilab.

All platforms show high stability in Monte Carlo simulations.

Differences in results are significant for certain matrix operations.

Abstract

In this article we test the accuracy of three platforms used in computational modelling: MatLab, Octave and Scilab, running on i386 architecture and three operating systems (Windows, Ubuntu and Mac OS). We submitted them to numerical tests using standard data sets and using the functions provided by each platform. A Monte Carlo study was conducted in some of the datasets in order to verify the stability of the results with respect to small departures from the original input. We propose a set of operations which include the computation of matrix determinants and eigenvalues, whose results are known. We also used data provided by NIST (National Institute of Standards and Technology), a protocol which includes the computation of basic univariate statistics (mean, standard deviation and first-lag correlation), linear regression and extremes of probability distributions. The assessment was made comparing the results computed by the platforms with certified values, that is, known results, computing the number of correct significant digits.