Concerning the Feasibility of Example-driven Modelling Techniques

TL;DR

This study evaluates the feasibility of example-driven modelling techniques compared to traditional spreadsheet methods by analyzing error rates, task complexity, and user confidence through controlled experiments.

Contribution

It provides empirical evidence on the relationship between error and task complexity in example-driven versus traditional modelling methods.

Findings

Example-driven techniques show different error patterns compared to traditional spreadsheets.

Task complexity impacts accuracy and confidence levels in modelling tasks.

Statistical tests confirm significant differences in performance variables.

Abstract

We report on a series of experiments concerning the feasibility of example driven modelling. The main aim was to establish experimentally within an academic environment: the relationship between error and task complexity using a) Traditional spreadsheet modelling; b) example driven techniques. We report on the experimental design, sampling, research methods and the tasks set for both control and treatment groups. Analysis of the completed tasks allows comparison of several different variables. The experimental results compare the performance indicators for the treatment and control groups by comparing accuracy, experience, training, confidence measures, perceived difficulty and perceived completeness. The various results are thoroughly tested for statistical significance using: the Chi squared test, Fisher's exact test for significance, Cochran's Q test and McNemar's test on difficulty.