Each word in the name describes part of the recipe:
- Generative: it produces new text instead of only labeling or sorting text.
- Pre-trained: it first learns general language patterns from a very large body of text, before any task-specific training. See pre-training.
- Transformer: it uses the transformer neural network architecture.
OpenAI introduced the approach in a 2018 paper and used the name for its model series, from GPT-1 through later versions. ChatGPT is a chat product built on GPT models that were further trained to follow instructions and hold conversations. In everyday speech “GPT” is sometimes used for any chatbot, but strictly it refers to OpenAI’s model family and the design behind it.
Why it matters
The GPT recipe, large-scale pre-training followed by fine-tuning or post-training, became the template for most modern large language models, including models from other companies that don’t use the GPT name.
Related terms
Large language model is the broader category, and foundation model is the even broader one.
Sources
- Radford et al. (OpenAI): Improving Language Understanding by Generative Pre-Training — The 2018 OpenAI paper describing generative pre-training of a transformer language model on unlabeled text, followed by discriminative fine-tuning on each task; checked October 24, 2026.
- Brown et al.: Language Models are Few-Shot Learners — The GPT-3 paper, showing that scaling a pre-trained transformer language model lets it perform many tasks from a few examples in the prompt; checked October 24, 2026.