A Geometric Method to Obtain the Generation Probability of a Sentence

TL;DR

This paper introduces a mathematical approach to estimate the probability of a sentence by analyzing individual word probabilities and their co-occurrences, reflecting how humans synthesize sentences.

Contribution

It proposes a novel geometric method to compute sentence generation probability based on word network co-occurrences, offering a new perspective in NLP modeling.

Findings

Sentence probability can be derived from word and word pair probabilities.

The method aligns with human sentence synthesis processes.

Experimental results support the effectiveness of the approach.

Abstract

"How to generate a sentence" is the most critical and difficult problem in all the natural language processing technologies. In this paper, we present a new approach to explain the generation process of a sentence from the perspective of mathematics. Our method is based on the premise that in our brain a sentence is a part of a word network which is formed by many word nodes. Experiments show that the probability of the entire sentence can be obtained by the probabilities of single words and the probabilities of the co-occurrence of word pairs, which indicate that human use the synthesis method to generate a sentence.