Network Models of Expertise in the Complex Task of Operating Particle Accelerators

TL;DR

This study models expertise in operating particle accelerators using network analysis of operator logs, revealing common strategies in task decomposition across different expertise levels.

Contribution

It introduces a novel network-based method to analyze how expertise develops in complex real-world tasks through natural language processing of operational logs.

Findings

Operators change their subtask focus with expertise

Subtask groupings are remarkably consistent across expertise levels

Operators adopt a divide-and-conquer approach to complex tasks

Abstract

We implement a network-based approach to study expertise in a complex real-world task: operating particle accelerators. Most real-world tasks we learn and perform (e.g., driving cars, operating complex machines, solving mathematical problems) are difficult to learn because they are complex, and the best strategies are difficult to find from many possibilities. However, how we learn such complex tasks remains a partially solved mystery, as we cannot explain how the strategies evolve with practice due to the difficulties of collecting and modeling complex behavioral data. As complex tasks are generally networks of many elementary subtasks, we model task performance as networks or graphs of subtasks and investigate how the networks change with expertise. We develop the networks by processing the text in a large archive of operator logs from 14 years of operations using natural language processing and machine learning. The network changes are examined using a set of measures at four levels of granularity - individual subtasks, interconnections among subtasks, groups of subtasks, and the whole complex task. We find that the operators consistently change with expertise at the subtask, the interconnection, and the whole-task levels, but they show remarkable similarity in how subtasks are grouped. These results indicate that the operators of all stages of expertise adopt a common divide-and-conquer approach by breaking the complex task into parts of manageable complexity, but they differ in the frequency and structure of nested subtasks. Operational logs are common data sources from real-world settings where people collaborate with hardware and software environments to execute complex tasks, and the network models investigated in this study can be expanded to accommodate multi-modal data. Therefore, our network-based approach provides a practical way to investigate expertise in the real world.