TL;DR
This paper introduces a hierarchical autoencoder with vector quantization for script generation, capturing complex hierarchical knowledge and outperforming previous language models in encoding and generating everyday scenario scripts.
Contribution
It presents a novel hierarchical quantized autoencoder model that effectively encodes and generates scripts, leveraging vector quantization to handle hierarchical knowledge structures.
Findings
Outperforms recent language modeling methods on standard script tasks.
Achieves lower perplexity scores than previous models.
Effectively encodes hierarchical script knowledge.
Abstract
Scripts define knowledge about how everyday scenarios (such as going to a restaurant) are expected to unfold. One of the challenges to learning scripts is the hierarchical nature of the knowledge. For example, a suspect arrested might plead innocent or guilty, and a very different track of events is then expected to happen. To capture this type of information, we propose an autoencoder model with a latent space defined by a hierarchy of categorical variables. We utilize a recently proposed vector quantization based approach, which allows continuous embeddings to be associated with each latent variable value. This permits the decoder to softly decide what portions of the latent hierarchy to condition on by attending over the value embeddings for a given setting. Our model effectively encodes and generates scripts, outperforming a recent language modeling-based method on several standard…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsSolana Customer Service Number +1-833-534-1729
