Decision Tree
A decision tree predicts by following a series of tests on input features, ending at a leaf with a class prediction or numerical value. Training chooses the splits from examples; the resulting paths can be read as if-then rules.
Imagine a small classifier trained to predict which support queue should receive a ticket. Its first test asks whether the ticket reports a payment problem. If yes, it predicts billing. Otherwise, a second test asks whether the customer can sign in, leading to technical support or general support. This is an illustrative tree, not a validated routing policy.
Training chooses tests that improve a criterion such as class purity or prediction error. A leaf can store a class distribution or a regression value; it need not represent certainty. Reading a path explains how this model produced a prediction, not what caused the customer’s problem.
Very deep trees can fit noise and become difficult to inspect. Limit depth or prune branches using validation, and compare with ensembles such as gradient boosting. The practical value is a traceable baseline: teams can see which features a prediction uses, then test whether those rules hold on new examples. Readability alone does not establish accuracy.
Sources
- scikit-learn: Decision trees — Explains learned feature splits, classification/regression leaves, overfitting and pruning.
Go deeper
- scikit-learn: Cost-complexity pruning docs
Compare tree size and held-out accuracy as pruning removes branches.