Feature Engineering

A subscription company wants to predict which customers will cancel next month. Its raw data is a pile of login events, support tickets and invoices. A model can’t use “a pile of events” directly, so someone builds features from it: logins in the last 14 days, change in usage compared with the previous month, number of open support tickets, days until renewal. Choosing and computing those inputs is feature engineering, and good choices often matter more than the choice of algorithm.

Common techniques include converting categories to numbers (one-hot encoding), putting numeric columns on a similar range (feature scaling), bucketing values, combining columns into ratios, and extracting parts of dates; related steps are feature selection and feature extraction. Google’s Rules of Machine Learning treats it as the main lever once a simple first model is in production. Every feature has to be computable at prediction time from data that exists then; using information from after the fact is data leakage.

Deep learning changed the balance: neural networks learn many of their own internal features from raw pixels, audio or text (representation learning). In episode 63 at 24:08, Yutori’s Dhruv Batra describes his team’s web agent model taking that route: it starts from pixels, “We’re not feature engineering,” and no human pre-designs how it reads web page structure. That is his account of one model’s design. For tabular business data such as spreadsheets and transaction logs, hand-built features remain common, and LLM applications still involve a similar job when deciding what goes into a prompt (context engineering).