A retailer trains a model to predict which shoppers will like a new product, using reviews from its website. Only people who bought the product and felt strongly enough to write a review are in the data, so the model learns from enthusiasts and complainers and misses the quiet majority. That is selection bias: the data was collected through a filter, and the filter shapes what the model learns.
Google’s machine learning course names three common forms. Coverage bias means some groups are missing, such as a speech model trained mostly on one accent. Non-response bias means the people who answered differ from those who didn’t, as with optional surveys. Sampling bias means the sample wasn’t random, for example taking the first thousand support tickets of the year and treating them as typical. Survivorship bias, studying only the companies or loans that lasted, is a related trap.
In AI, selection bias shows up in two places. In training data, it produces models that work well for the people and cases that were well represented and worse for everyone else, one route to unfair outcomes discussed under algorithmic fairness. In AI evaluation, a test set chosen from easy or convenient examples makes a system look better than it will perform in production, which is why a golden dataset should be drawn from real traffic and refreshed. Checking for it means comparing who or what is in the data with who or what the system will actually serve, and measuring performance separately for each group.
Sources
- Google Machine Learning Crash Course: Types of bias — Defines selection bias as choosing a dataset's examples in a way that does not reflect their real-world distribution, with coverage bias, non-response bias and sampling bias as forms; checked October 24, 2026.
- NIST SP 1270: Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022) — NIST's framework for statistical, human and systemic bias in AI, including bias from how data is selected and collected; checked October 24, 2026.