Perceptron
A perceptron is a simple classifier that multiplies inputs by weights, adds a bias and applies a threshold to produce a class label. The basic single-unit perceptron draws a linear decision boundary in its input features.
Consider two switches represented by inputs 0 or 1. With both weights set to 1 and a bias of −1.5, the score is x₁ + x₂ − 1.5. Output 1 when that score is positive and 0 otherwise. Only two on switches give a positive score, so this perceptron computes the logical AND operation.
In the basic learning rule, a misclassified example triggers an adjustment to the weights and bias. The classifier can learn a separating boundary when the examples are linearly separable. If they are not, that rule does not guarantee training will reach zero errors.
The limit is visible in exclusive OR (XOR): the label is 1 when exactly one switch is on. No single straight line separates those cases in the original two-input plane. Additional nonlinear features or hidden layers can represent a boundary the plain perceptron cannot. This matters when distinguishing a single perceptron from a multilayer perceptron, which combines units and nonlinear transformations.
Sources
- Jurafsky and Martin: Neural Networks — the XOR problem — Defines the threshold perceptron and explains why a single linear unit cannot represent XOR.
- scikit-learn: Perceptron — Describes the basic mistake-driven update and its perceptron-loss implementation.
Go deeper
- 3Blue1Brown: But what is a neural network? video
Visualize weights and activations, then see why combining units matters.