Support Vector Machine
- Definition
- Support Vector Machines (SVMs) try to find the boundary that separates classes with the largest margin. They can use kernels to operate in transformed feature spaces for non-linear separation. Picture fitting a fence that leaves the widest gap between classes. SVMs differ from probabilistic classifiers (like logistic regression) by focusing on margin maximization and from tree-based methods in how they handle feature interactions and generalization.
How does it work?
Support Vector Machine models learn from labeled examples: prepare features, choose a model family, train on examples, and validate on held-out data. Pay attention to data preprocessing, class imbalance, and hyperparameter tuning.
Examples
- Text classification — Linear SVMs on TF-IDF features for spam or sentiment classification.
- Face recognition (embedding classification) — Use kernel SVMs on precomputed embeddings for small-scale recognition tasks.
- Anomaly boundary detection — Use one-class SVM to model normal behaviour and detect outliers.
Problems
- Doesn't scale well to very large datasets
- Choosing the right kernel and its parameters is non-trivial
- Sensitive to feature scaling
- Limited interpretability with non-linear kernels
- Struggles with heavily overlapping or noisy classes