Imagine sorting email using two measurements: how many links a message contains and what share of its words are in capital letters. Plot past emails as dots, spam in red and normal mail in blue. A simple linear classifier, such as logistic regression, draws one straight line between the colors; everything on one side is called spam. A decision tree draws a staircase of horizontal and vertical steps, and k-nearest neighbors draws a wiggly border that follows the nearby dots. Each line is that model’s decision boundary.
With dozens or thousands of inputs you can’t draw the boundary, but the idea is the same: a surface in the space of inputs where the predicted class changes. Looking at boundaries on small examples is a useful way to compare models. A boundary that twists around individual training points is a sign of overfitting. A support vector machine deliberately places its boundary as far as possible from the nearest examples of each class.
A boundary shows what the model learned, not what is true in the world. Inputs far from any training data still land on one side of it, so the model may give confident answers where it has no evidence. Distance from the boundary is also not automatically a trustworthy probability; that needs calibration.
Sources
- scikit-learn: Classifier comparison — Plots the decision boundaries of nearest neighbors, linear and RBF support vector machines, decision trees, random forests, neural networks and other classifiers on the same toy datasets; checked October 24, 2026.
- scikit-learn: Support Vector Machines — Describes support vector classifiers as separating classes with a hyperplane and exposing a decision function for each sample; checked October 24, 2026.