AI Engineer

Picture a support team that wants an assistant to answer refund questions. An AI engineer picks a model, writes the system prompt, connects the help-center articles through retrieval-augmented generation, gives the assistant a tool to look up orders, and builds an evaluation set to check that answers stay correct as the prompt or model changes. Nobody on that team trains a model.

The title spread in 2023, when Latent Space argued that foundation models had moved a lot of applied AI from research teams to software engineers working through an API. A machine learning engineer usually trains, tunes and deploys models on an organization’s own data; an AI engineer usually starts from someone else’s model. The line moves both ways: AI engineers fine-tune and host open models, and ML engineers build on APIs.

The show explores which skills the role requires. In episode 37 at 18:23, Galileo co-founder Atindriyo Sanyal says the main skill a software engineer needs to become an AI engineer is data literacy: telling good data from bad. Hamel Husain adds at 17:21 that the work quickly arrives at the skills of an ML engineer or data scientist, even without training models, and predicts at 20:29 that the role may split into sub-disciplines. Episode 37 was sponsored by Galileo; Sanyal is introduced as its CTO and the show’s co-host at 0:35.

In episode 38 at 39:09, Aurimas Griciūnas rejects the idea that AI engineering is just data engineering: data engineering pipes data to where it needs to live, while AI engineering builds agentic systems on top of it (39:41). The episode was sponsored by Galileo, which Conor plugs at 28:13.

Titles vary between companies, so read a job description for the actual work. A useful test is whether the role is judged on model quality (closer to ML engineering) or on whether a product built with models works for its users (closer to AI engineering). See also product engineer and LLMOps.