Natural language processing

Field of artificial intelligence and machine learning, meant to analyse textual data.

I was first exposed to this as a research assistant with the Discovery programme.

Basics

Language models in general learn a probability distribution over a sequence of words. They have the goal of text understanding (question understanding, sentiment analysis) and text generation (sentence completion, generating captions, translation).

The problem with working with text compared to working with images is: grammar, spelling, that there are many words to learn, choosing between working with words and characters, arbitrary length IO, different languages.

Key concepts

• Text pre-processing
  • Tokenisation
  • Lemmatisation
• Semantic coherence
• Word structure
  • n-gram
  • Vector embedding
    • word2vec
    • GloVe
• Association rule mining
• Large language model
• Applications
  • Sentiment analysis
  • Thematic analysis
  • Machine translation

Tools

• NLTK
• spaCy
• torchtext

Resources

• Natural Language Processing with Python, by Steven Bird, Ewan Klein, and Edward Loper
• Natural Language Processing and Computational Linguistics, by Bhargav Srinivasa-Desikan
• CSC401 — Natural Language Computing course notes by Prof Frank Rudzicz

See also

• Computational linguistics