TL;DR
This paper introduces two models, LSTM-CRF and BERT-LSTM-CRF, for semantic tagging, comparing their convergence and dataset requirements on a universal semantic tag dataset.
Contribution
It presents and evaluates two semantic tagging models, highlighting the ease of convergence of LSTM-CRF and the data-intensive nature of BERT-LSTM-CRF.
Findings
LSTM-CRF converges faster and is easier to train.
BERT-LSTM-CRF requires larger datasets and longer training time.
BERT-LSTM-CRF achieves better performance with sufficient data.
Abstract
In the present paper, two models are presented namely LSTM-CRF and BERT-LSTM-CRF for semantic tagging of universal semantic tag dataset. The experiments show that the first model is much easier to converge while the second model that leverages BERT embedding, takes a long time to converge and needs a big dataset for semtagging to be effective.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Text and Document Classification Technologies
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Linear Layer · Adam · Layer Normalization · Weight Decay · Multi-Head Attention · Residual Connection · Dense Connections · Dropout