Ordered Memory Baselines
Daniel Borisov, Matthew D'Iorio, Jeffrey Hyacinthe

TL;DR
This paper reviews the Ordered Memory model for natural language processing, demonstrating it performs comparably to state-of-the-art tree models and outperforms simpler, parameter-efficient baselines.
Contribution
It provides a baseline comparison for the Ordered Memory model, highlighting its effectiveness and efficiency relative to existing models.
Findings
Ordered Memory matches state-of-the-art performance
Simpler baselines require fewer parameters but perform worse
Ordered Memory is effective for tree-structured semantic tasks
Abstract
Natural language semantics can be modeled using the phrase-structured model, which can be represented using a tree-type architecture. As a result, recent advances in natural language processing have been made utilising recursive neural networks using memory models that allow them to infer tree-type representations of the input sentence sequence. These new tree models have allowed for improvements in sentiment analysis and semantic recognition. Here we review the Ordered Memory model proposed by Shen et al. (2019) at the NeurIPS 2019 conference, and try to either create baselines that can perform better or create simpler models that can perform equally as well. We found that the Ordered Memory model performs on par with the state-of-the-art models used in tree-type modelling, and performs better than simplified baselines that require fewer parameters.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Sentiment Analysis and Opinion Mining · Natural Language Processing Techniques
