Autoencoder
- Definition
- An Autoencoder compresses input into a compact latent representation and decodes it to reconstruct the original. It's used for dimensionality reduction, denoising, and representation learning. Picture an encoder that translates data into a smaller code and a decoder that reconstructs it; training minimizes reconstruction error. Autoencoders differ from supervised models because they don't need labels, and from VAEs because standard autoencoders don't impose probabilistic structure on the latent space.
How does it work?
Autoencoder architectures are specialised neural designs; they define layer types and connectivity that bias the model to particular data structures. Implementations balance depth, width, and compute considerations for practical training.
Examples
- Dimensionality reduction — Learn compact latent codes for visualization or fast retrieval.
- Anomaly detection — Train autoencoders to reconstruct normal examples; high reconstruction error flags anomalies.
- Image compression — Learn lossy compression codecs by minimising reconstruction error in the latent space.
Problems
- Can learn a trivial identity mapping without proper regularization
- Latent space may not be smooth or meaningfully structured
- Reconstruction quality vs. compression tradeoff is hard to balance
- Sensitive to choice of bottleneck size and architecture