WISDOM: An AI-powered framework for emerging research detection using weak signal analysis and advanced topic modeling

TL;DR

WISDOM is an AI-driven framework that leverages weak signal analysis and advanced topic modeling to automatically detect and track emerging research trends across disciplines, improving over traditional bibliometric methods.

Contribution

This work introduces WISDOM, a novel automated framework combining weak signal analysis and advanced topic modeling for unbiased detection of emerging scientific topics.

Findings

Successfully identified emerging underwater sensing research trends from 2004 to 2021.

Outperformed traditional bibliometric methods in detecting interdisciplinary research signals.

Enhanced detection accuracy and objectivity in trend analysis.

Abstract

The landscape of science and technology is characterized by its dynamic and evolving nature, constantly reshaped by new discoveries, innovations, and paradigm shifts. Moreover, science is undergoing a remarkable shift towards increasing interdisciplinary collaboration, where the convergence of diverse fields fosters innovative solutions to complex problems. Detecting emerging scientific topics is paramount as it enables industries, policymakers, and innovators to adapt their strategies, investments, and regulations proactively. As the common approach for detecting emerging technologies, despite being useful, bibliometric analyses may suffer from oversimplification and/or misinterpretation of complex interdisciplinary trends. In addition, relying solely on domain experts to pinpoint emerging technologies from science and technology trends might restrict the ability to systematically analyze extensive information and introduce subjective judgments into the interpretations. To overcome these drawbacks, in this work, we present an automated artificial intelligence-enabled framework, called WISDOM, for detecting emerging research themes using advanced topic modeling and weak signal analysis. The proposed approach can assist strategic planners and domain experts in more effectively recognizing and tracking trends related to emerging topics by swiftly processing and analyzing vast volumes of data, uncovering hidden cross-disciplinary patterns, and offering unbiased insights, thereby enhancing the efficiency and objectivity of the detection process. As the case technology, we assess WISDOM's performance in identifying emerging research as well as their trends, in the field of underwater sensing technologies using scientific papers published between 2004 and 2021.