Multitask learning

A self-driving perception model may need to find lanes, spot pedestrians and read traffic signs. Training three separate models wastes effort, because all three tasks depend on understanding the same road scene. A multitask model shares most of its layers across the tasks and adds a small output “head” for each one. Learning to find pedestrians also teaches the shared layers about edges, depth and motion, which helps with lanes too.

Where the idea comes from. Rich Caruana’s 1997 paper argued that related tasks act as extra training signal for each other, because the shared layers have to learn features that work for all of them. Sebastian Ruder’s survey describes the two common designs: hard parameter sharing, where tasks share the same hidden layers, and soft sharing, where each task has its own model but the models are pushed to stay similar.

Business example. A customer-support model can be trained at once to classify the ticket’s topic, estimate urgency and detect sentiment. One model is cheaper to serve than three, and the shared training can help the task with the least labeled data.

When it hurts. Tasks that pull in different directions can interfere, so one improves while another gets worse. Teams then have to weight the tasks or split them back apart. Editorial synthesis: large language models are in effect a very broad form of multitask learning, since instruction tuning mixes many tasks into one training run, and the same interference shows up when a fine-tuning run for one skill degrades another.

Why it matters. Multitask learning is a core reason a single foundation model can serve many products. See also transfer learning and meta-learning.