Autoencoder
An autoencoder is a neural network trained to encode an input into an internal code and reconstruct the input from that code. A smaller bottleneck is one design; other autoencoders use constraints or regularization to learn useful representations.
An encoder maps an input to a code, and a decoder maps that code back to a reconstruction. Training penalizes differences between the reconstruction and its target. The internal representation belongs to a latent space; its coordinates do not automatically have human-readable meanings.
For example, a basic autoencoder can turn a 28-by-28 grayscale digit image into 32 numbers, then reconstruct 784 pixel values. A denoising version instead receives a corrupted image and learns to reconstruct a clean target. Neither setup requires a digit-category label for every image.
A bottleneck or suitable constraint matters because simply copying the input can be an easy but unhelpful solution. Reconstruction error can help flag unusual inputs, but a high error is not proof of a defect, and a low error is not proof of normality. Check it against the actual task. An autoencoder’s reconstruction objective also differs from an embedding model trained specifically to rank relevant search results.
Sources
- Goodfellow, Bengio and Courville: Autoencoders — Explains reconstruction objectives, undercomplete and regularized designs, and denoising autoencoders.
Go deeper
- Keras: Convolutional autoencoder for image denoising docs
Build an encoder and decoder, then compare noisy inputs with reconstructed digit images.