Data Science

A retailer asks why customers in one region cancel subscriptions more often. A data scientist pulls the account and support data, checks it for gaps, tests a few explanations (price changes, delivery delays, a competitor’s launch), and reports which one the evidence supports and how confident the answer is. Sometimes the output is a chart and a recommendation; sometimes it is a machine learning model that predicts which customers are likely to cancel.

Data science overlaps with statistics, analytics and data mining. NIST’s definition describes it as a superset that also includes managing the end-to-end data process. In practice, a data scientist usually answers questions and builds prototypes; a machine learning engineer makes models run reliably in production.

Generative AI pulled data-science habits into software teams. In episode 5 at 2:00, Galileo CEO Vikram Chatterji says evaluation was mostly done by data scientists, and that software engineers calling models through an API now need the same test sets, metrics and experiments. His own estimate is that there are about 500,000 data scientists and 30 million developers. Episode 5 was sponsored by Galileo; its read is at 20:00.

In episode 66 at 20:24, Starburst’s Jitender Aswani explains that data scientist and analyst roles existed partly because someone had to know what data an organization had and what shape it was in. He argues that natural-language interfaces over well-curated data let business users ask those questions directly. That is his forecast; it still depends on the curation work he describes.

See also AI engineer, dataset and training data.