Enhanced Integrated Scoring for Cleaning Dirty Texts
Wilson Wong, Wei Liu, Mohammed Bennamoun

TL;DR
This paper improves the ISSAC system for cleaning and preprocessing dirty texts from online sources, achieving higher accuracy in spelling correction, abbreviation expansion, and case restoration for ontology engineering.
Contribution
The paper introduces an enhanced version of ISSAC that significantly improves text preprocessing accuracy in ontology engineering applications.
Findings
Achieved 98% accuracy with enhanced ISSAC on chat records.
Outperformed basic ISSAC and Aspell in cleaning accuracy.
Demonstrated effectiveness on 700 chat records.
Abstract
An increasing number of approaches for ontology engineering from text are gearing towards the use of online sources such as company intranet and the World Wide Web. Despite such rise, not much work can be found in aspects of preprocessing and cleaning dirty texts from online sources. This paper presents an enhancement of an Integrated Scoring for Spelling error correction, Abbreviation expansion and Case restoration (ISSAC). ISSAC is implemented as part of a text preprocessing phase in an ontology engineering system. New evaluations performed on the enhanced ISSAC using 700 chat records reveal an improved accuracy of 98% as compared to 96.5% and 71% based on the use of only basic ISSAC and of Aspell, respectively.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Algorithms and Data Compression · Web Data Mining and Analysis
