Natural Language Processing (almost) from Scratch

TL;DR

This paper introduces a versatile neural network architecture that learns internal representations from large unlabeled datasets, enabling it to perform multiple NLP tasks without task-specific engineering.

Contribution

The work presents a unified neural approach that minimizes task-specific feature engineering, relying instead on learned representations from unlabeled data for various NLP tasks.

Findings

Achieved good performance across multiple NLP tasks

Reduced need for task-specific feature engineering

Built a computationally efficient, freely available tagging system

Abstract

We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.