GAMMT: Generative Ambiguity Modeling Using Multiple Transformers

TL;DR

GAMMT introduces a novel sequence modeling approach using multiple transformers to capture ambiguity in data generation, aiming for high quality and diverse sequence representations.

Contribution

The paper proposes GAMMT, a new model employing multiple transformers linked by a selection mechanism to model ambiguous probabilistic data sequences.

Findings

Conceptual framework for ambiguity modeling

Potential for high-quality, diverse sequence generation

No experimental validation yet

Abstract

We introduce a novel model called GAMMT (Generative Ambiguity Models using Multiple Transformers) for sequential data that is based on sets of probabilities. Unlike conventional models, our approach acknowledges that the data generation process of a sequence is not deterministic, but rather ambiguous and influenced by a set of probabilities. To capture this ambiguity, GAMMT employs multiple parallel transformers that are linked by a selection mechanism, allowing for the approximation of ambiguous probabilities. The generative nature of our approach also enables multiple representations of input tokens and sequences. While our models have not yet undergone experimental validation, we believe that our model has great potential to achieve high quality and diversity in modeling sequences with uncertain data generation processes.