NOVAID: Natural-language Observability Visualization Assistant for ITOps Dashboard Widget Generation

TL;DR

NOVAID is an AI-powered chatbot that enables users to generate IT monitoring dashboard widgets from natural language, improving speed, accuracy, and usability for IT operations teams.

Contribution

The paper introduces NOVAID, a novel LLM-based system tailored for IT dashboard widget generation, addressing domain-specific challenges and enhancing usability.

Findings

Achieves up to 94.10% accuracy in widget specification extraction.

User study reports a System Usability Scale score of 74.2.

Demonstrates effective handling of complex IT-specific visualization tasks.

Abstract

Manual creation of IT monitoring dashboard widgets is slow, error-prone, and a barrier for both novice and expert users. We present NOVAID, an interactive chatbot that leverages Large Language Models (LLMs) to generate IT monitoring widgets directly from natural language queries. Unlike general natural language-to-visualization tools, NOVAID addresses IT operations-specific challenges: specialized widget types like SLO charts, dynamic API-driven data retrieval, and complex contextual filters. The system combines a domain-aware semantic parser, fuzzy entity matching, and schema completion to produce standardized widget JSON specifications. An interactive clarification loop ensures accuracy in underspecified queries. On a curated dataset of 271 realistic queries, NOVAID achieves promising accuracy (up to 94.10% in metric extraction) across multiple LLMs. A user study with IT engineers yielded a System Usability Scale score of 74.2 for NOVAID, indicating good usability. By bridging natural language intent with operational dashboards, NOVAID demonstrates clear potential and a path for deployment in enterprise ITOps monitoring platforms.