Man vs machine: an experimental study of geosteering decision skills

TL;DR

This study compares geosteering experts with an automatic decision support system in a synthetic environment, finding the system consistently outperforms human experts in decision-making tasks.

Contribution

It provides the first controlled experiment demonstrating the superior and consistent performance of an automated decision support system over human experts in geosteering.

Findings

DSS-1 outperformed 28 of 29 geoscientists.

No expert beat DSS-1 more than once.

DSS-1 showed consistent decision-making performance.

Abstract

With the steady growth of the amount of real-time data while drilling, operational decision-making is becoming both better informed and more complex. Therefore, as no human brain has the capacity to interpret and integrate all decision-relevant information from the data, the adoption of advanced algorithms is required not only for data interpretation but also for decision optimization itself. However, the advantages of the automatic decision-making are hard to quantify. The main contribution of this paper is an experiment in which we compare the decision skills of geosteering experts with those of an automatic decision support system in a fully controlled synthetic environment. The implementation of the system, hereafter called DSS-1, is presented in our earlier work [Alyaev et al. "A decision support system for multi-target geosteering." Journal of Petroleum Science and Engineering 183 (2019)]. For the current study we have developed an easy-to-use web-based platform which can visualize and update uncertainties in a 2D geological model. The platform has both user and application interfaces (GUI and API) allowing us to put human participants and DSS-1 into a similar environment and conditions. The results of comparing 29 geoscientists with DSS-1 over three experimental rounds showed that the automatic algorithm outperformed 28 participants. What's more, no expert has beaten DSS-1 more than once over the three rounds, giving it the best comparative rating among the participants. By design DSS-1 performs consistently, that is, identical problem setup is guaranteed to yield identical decisions. The study showed that only two experts managed to demonstrate partial consistency within a tolerance but ended up with much lower scores.