The first wave was autocomplete: a model suggests the next line as you type. Then chat, where you describe a function and paste the result. Now a coding agent can take an issue, edit several files, run the tests and open a pull request. All of these are AI code generation; they differ in how much the model decides on its own.
Benchmarks track progress by running what the model writes. HumanEval checks whether small standalone functions are functionally correct by running them against tests; SWE-bench checks patches to real repositories against their test suites. The work after generation includes review, deployment and maintenance, the subject of who reviews AI-generated code. When the person prompting doesn’t read the output closely, the practice is usually called vibe coding.
In episode 18 at 16:30, Honeycomb’s Charity Majors argues that writing greenfield code is typically the fastest, easiest part of the software lifecycle, which is why new technology reaches for code generation first. The episode was sponsored by Galileo (mention at 41:05).
Graphite’s Greg Foster picks up from there in episode 40 at 14:39, walking through what happens after code generation on a local machine: pull request, review and merge. The episode was sponsored by Galileo (mention at 2:05).
Hear it from the guest
“Greenfield code is typically the fastest, easiest part of the lifecycle.”
Quotes lightly edited to remove filler words.
Sources
- Chen et al.: Evaluating Large Language Models Trained on Code (arXiv 2107.03374) — The Codex paper introduces HumanEval, an evaluation set that measures the functional correctness of Python programs synthesized from docstrings; checked October 24, 2026.
- Jimenez et al.: SWE-bench, Can Language Models Resolve Real-World GitHub Issues? (arXiv 2310.06770) — Benchmark of real GitHub issues that measures whether a model's generated patch makes the repository's tests pass; checked October 24, 2026.
