Logistic Regression

A bank wants to estimate the chance a loan applicant will default. Logistic regression multiplies each input (income, debt ratio, payment history) by a weight and adds them up, just as linear regression does. Then it passes the total through an S-shaped curve called the sigmoid, which turns any number into a probability between 0 and 1. A score of 0.08 means “about an 8% chance.” Pick a threshold, say 0.5, and the probability becomes a yes-or-no decision.

The name is a historical quirk: the model regresses the log-odds of an outcome, but in practice it’s a classification tool, as scikit-learn notes. Versions with several categories exist too. It remains common in credit scoring, medical risk scores, marketing response models and as a first baseline for any classifier, because it is fast, needs little data and its weights can be explained to a regulator or a doctor.

Its probabilities are only useful if they match reality (calibration), and its errors are best read through a confusion matrix and precision and recall, since moving the threshold trades false positives against false negatives. Like linear regression, it assumes a fairly straight-line relationship (in log-odds) and needs help from feature engineering to capture curves. The final layer of many neural network classifiers performs a similar calculation. In episode 42 at 48:49, Aishwarya Srinivasan uses logistic regression as her example of a simple, traditional model when arguing that evaluation challenges with AI agents echo older machine learning ones; that comparison is her view.