Random Forest
- Definition
- Random Forest builds many decision trees on random subsets of data and features, then averages their predictions. This reduces overfitting and typically improves accuracy compared to a single tree. S, imagine asking many slightly different experts and taking a majority vote. Random Forest differs from boosting because it trains trees independently and aggregates, while boosting adds trees sequentially to correct previous errors.
How does it work?
Trees split data by feature thresholds to partition space; ensemble methods build many trees on bootstrapped samples and aggregate predictions. Focus on feature selection, depth control, and understanding variable importance when using these models.
Examples
- Fraud detection — Ensemble tree models detect anomalous transactions using many randomised trees.
- Feature importance analysis — Use permutation importance from forests to prioritise variables for product teams.
- Retail demand forecasting — Apply ensembles on tabular sales data for robust short-term forecasts.
Problems
- Large ensembles are memory- and compute-heavy at inference time
- Reduced interpretability compared to a single tree
- Can still overfit noisy data with too many deep trees
- Biased toward features with many categories in importance rankings