Field of artificial intelligence and machine learning, meant to analyse textual data.
I was first exposed to this as a research assistant with the Discovery programme.
Basics
Language models in general learn a probability distribution over a sequence of words. They have the goal of text understanding (question understanding, sentiment analysis) and text generation (sentence completion, generating captions, translation).
The problem with working with text compared to working with images is: grammar, spelling, that there are many words to learn, choosing between working with words and characters, arbitrary length IO, different languages.
Key concepts
- Text preprocessing
- Semantic coherence
- Word structure
- Association rule mining
- Large language model
- Applications
Tools
Resources
- Natural Language Processing with Python, by Steven Bird, Ewan Klein, and Edward Loper
- Natural Language Processing and Computational Linguistics, by Bhargav Srinivasa-Desikan
- CSC401 — Natural Language Computing course notes by Prof Frank Rudzicz