Data Flywheel

A navigation app is the classic example. Every driver’s phone reports speed and location, the app uses that data to predict traffic better, better predictions bring in more drivers, and more drivers mean more data. A support chatbot can work the same way: conversations where people rate an answer or a human agent corrects it become examples for fine-tuning or evaluation, which improves the next round of answers.

The loop is why companies talk about data as a moat. A competitor can copy a feature or license the same foundation model, but not years of feedback from real users in a specific domain.

The flywheel doesn’t spin on its own. Raw logs need cleaning, labeling and consent; feedback can be biased toward the users who bother to give it; and a model trained mostly on its own outputs can degrade. Our editorial view: the claim that “more usage means a better model” only holds if a team actually closes the loop with good training data and evaluation. Many products collect data that never feeds back into anything.

On the show, the idea comes up with expert feedback rather than consumer clicks. In episode 69, at 51:39, Thomson Reuters’ Joel Hron calls the company’s legal model release “a compounding flywheel of benefit”: each round of work with in-house legal experts makes the next model better. That is his description of the company’s strategy. Episode 69 credits Svix as a Season 4 presenting sponsor in its sponsor read at 27:04 and Walrus Memory in a second read at 44:12.

For preparing that feedback for training, see How to Prepare Data for Fine-Tuning an LLM. Related: synthetic data.