Generative vs. discriminative models

Take a pile of animal photos labeled “cat” or “dog”. A discriminative model learns what separates the two: given a photo, is it more likely a cat or a dog? A generative model learns what cats and dogs look like in general, well enough to draw a new, plausible cat. Google’s explanation puts it in probability terms. A generative model captures how likely the data is, p(X) or p(X, Y). A discriminative model captures only how likely each label is given the data, p(Y | X).

Business examples. A fraud score on a card payment, a spam filter and a churn predictor are usually discriminative. They answer one narrow question about an input. A chatbot drafting an email, an image generator and a variational autoencoder are generative, because they produce new content. A large language model is generative because it models the probability of text, which is why it can write rather than only classify.

Trade-offs. Discriminative models solve a narrower problem, so they are often cheaper, faster and more accurate on that one job. Generative models can do more, and today’s large content-generating models need far more data and compute than a typical classifier, though that isn’t true of every generative model. Ng and Jordan’s 2001 paper found that a simple generative classifier can do better with little data, while the discriminative one usually wins as data grows.

Why it matters. The line has blurred: many teams now use a generative LLM for classification by prompting it to output a label. Editorial synthesis: that is flexible, but for a high-volume task with fixed labels, a small discriminative model, such as a fine-tuned BERT classifier, can be cheaper and faster. “Do we need to generate anything here?” is a useful first question when choosing a model.