Fr\'{e}chet Power-Scenario Distance: A Metric for Evaluating Generative AI Models across Multiple Time-Scales in Smart Grids

TL;DR

This paper introduces a new Fréchet-based metric to evaluate the quality of synthetic data generated by AI models in smart grids, addressing limitations of traditional pairwise distance measures across multiple time-scales.

Contribution

It proposes a novel Fréchet Power-Scenario Distance metric that assesses generative model quality from a distributional perspective in learned feature space.

Findings

The metric outperforms traditional methods across various timescales.

It improves the reliability of synthetic data evaluation in smart grid applications.

Empirical results confirm its effectiveness in different generative models.

Abstract

Generative artificial intelligence (AI) models in smart grids have advanced significantly in recent years due to their ability to generate large amounts of synthetic data, which would otherwise be difficult to obtain in the real world due to confidentiality constraints. A key challenge in utilizing such synthetic data is how to assess the data quality produced from such generative models. Traditional Euclidean distance-based metrics only reflect pair-wise relations between two individual samples, and could fail in evaluating quality differences between groups of synthetic datasets. In this work, we propose a novel metric based on the Fr\'{e}chet Distance (FD) estimated between two datasets in a learned feature space. The proposed method evaluates the quality of generation from a distributional perspective. Empirical results demonstrate the superiority of the proposed metric across timescales and models, enhancing the reliability of data-driven decision-making in smart grid operations.