Few-Shot Learning

Few-shot learning describes a task with only a small number of examples; few-shot prompting is one way to use those examples. Google’s glossary distinguishes the broader learning problem from the prompting technique (Google, Few-shot learning). In Brown and colleagues’ GPT-3 experiments, demonstrations went into the input context without gradient updates or fine-tuning (Language Models are Few-Shot Learners).

For example, give a ticket classifier two demonstrations: “charged twice” maps to billing, and “cannot sign in” maps to account access. Then ask it to classify “password reset link expired.” That is few-shot prompting. The demonstrations help specify the task for that request; they do not create a new trained checkpoint. Keep test tickets out of the demonstrations so you can assess the change on unseen inputs.

In a related healthcare discussion, Andreas Cleve contrasts one-shot or few-shot documentation with what he calls a recursive reasoning pipeline in episode 34 (38:45).

His point is about that workflow, rather than evidence for a general few-shot learning result. For builders choosing prompts, compare the same held-out tasks with and without examples and inspect mistakes by category. Context window covers the space the demonstrations occupy; fine-tuning changes weights instead. How to brief an AI assistant covers making the task and expected result explicit.

Anthropic’s prompting guide also calls this multishot prompting and recommends examples that mirror the real use case and cover varied cases (Anthropic, Prompting best practices). The underlying ability to pick up a task from examples in the input is called in-context learning; prompt covers what else goes into a model’s input.