Before you buy a used car, you look at the spec sheet and the service history. A model card is that document for an AI model. If a company is choosing between two open models for summarizing support tickets, the model cards tell it which languages each was trained on, which tests they were scored on, what the license allows and what the developers say the model should not be used for.
The idea comes from a 2018 paper by Margaret Mitchell and colleagues at Google. They proposed that trained models ship with short documents reporting intended use, the evaluation procedure, and performance broken down by relevant conditions and groups, such as skin type or age for a face-analysis model, so users can see where it works less well (Mitchell et al.). The format spread. On Hugging Face, the model card is the README file in every model repository and is expected to cover intended uses, limitations and biases, training data and evaluation results (Hugging Face). Large labs publish model cards or longer “system cards” alongside new releases, often with safety testing results.
Model cards matter for two reasons. For AI governance, they are a record of what a model was meant to do and what its makers knew about its limits, which buyers and regulators can check against. For engineering, they are usually the first place you see a model’s benchmark scores. But the numbers are reported by the people releasing the model, on tests they chose, and benchmark questions can leak into training data.
In episode 49 at 18:38, Intercom’s Fergal Reid describes exactly that balance: “we look at the model cards,” then worry about benchmark contamination and overfitting, so Intercom also runs its own back tests built from real hallucinations customers reported. Episode 49 is sponsored by Galileo, the Season 3 presenting sponsor; the read is at 28:52. The editorial takeaway: read the model card to shortlist models and learn their stated limits, then run your own AI evaluation on your own data before you decide.
Hear it from the guest
“In terms of external benchmarks, like everybody else, we look at the model cards that release and you get some sort of a sense there, but then you also have to worry about like contamination or people like overfitting to the benchmarks.”
Quotes lightly edited to remove filler words.
Sources
- Mitchell et al., Model Cards for Model Reporting — The 2018 paper that proposed model cards: short documents accompanying trained models that report benchmarked evaluation across relevant conditions and demographic groups, intended use, and evaluation procedure; checked October 24, 2026.
- Hugging Face Hub documentation: Model Cards — Explains that a model card is the README.md in a model repository and should describe intended uses, limitations and biases, training parameters, training datasets and evaluation results; checked October 24, 2026.
- Chain of Thought, episode 49: Fergal Reid on model cards and benchmarks — Reid says Intercom reads published model cards for external benchmarks but worries about contamination and overfitting, so it relies on its own back tests.
