Conditional Random Field
- Definition
- A Conditional Random Field (CRF) models the conditional distribution of label sequences given inputs, capturing dependencies between neighboring labels (useful in sequence labeling). Think of labeling a sentence where the label for a word depends on nearby labels as well as the input features. CRFs differ from HMMs by being discriminative (modeling p(labels|inputs) directly) and from simple classifiers by modeling structured outputs jointly.
How does it work?
Conditional Random Field algorithms typically represent uncertainty explicitly and use probability calculations to combine data and model assumptions. Implementations focus on numerical stability, how to represent distributions (parametric vs sampled), and efficient updates when new data arrive.
Examples
- Sequence labelling (NER) — Tag tokens in sentences with labels by modelling conditional dependencies across labels.
- Part-of-speech tagging — Model label interactions to improve tagging accuracy over independent classifiers.
- Handwriting recognition post-processing — Refine per-character predictions using CRF smoothing.
Problems
- Training is computationally expensive compared to simpler classifiers
- Feature engineering is often needed to get good performance
- Inference (decoding) can be slow on long sequences
- Scaling to large label sets increases complexity significantly