A neural network takes numbers in, passes them through layers of units that each combine their inputs using learned weights, and produces numbers out. Training works by showing the network a batch of examples, measuring how wrong its outputs are, and nudging the weights slightly in the direction that would have made them less wrong. Repeated over a huge number of examples, that process lets the network capture patterns no one wrote down.
The name comes from a loose analogy to neurons in the brain, but modern networks are better understood as very large mathematical functions. The weights are the model’s parameters. The simplest layered design is the multilayer perceptron; a recurrent neural network adds a state that carries over between steps of a sequence. The transformer is the neural network design behind today’s large language models. How large language models actually work shows where the network fits in.
One unit, worked by hand
A teaching example, not a trained model: one unit receives inputs 2 and 3, with weights 0.5 and −1 and a bias of 1. Its weighted sum is 2 × 0.5 + 3 × (−1) + 1 = −1. A ReLU activation turns any negative value into 0, so the unit passes 0 to the next layer. Training nudges the two weights and the bias so that, across many examples, outputs like this one move toward the right answers. A large network repeats this arithmetic billions of times per input.
Sources
- LeCun, Bengio and Hinton: Deep learning — Describes deep learning as models composed of multiple processing layers that learn representations of data, trained by backpropagation adjusting internal parameters; checked October 25, 2026.
Go deeper
From the conversation
-
Agent Memory: The Last Battleground in the AI Stack | Richmond Alake, Oracle -
First Code, Then AGI: Software’s Event Horizon with Poolside Founders Jason Warner & Eiso Kant -
I Started r/AI_Agents and Now I'm Launching a VC Fund | Yujian Tang -
Tuning GPU Performance with AI Agents | AMD’s Anush Elangovan on ROCm 10 -
Why LLMs Are Plausibility Engines, Not Truth Engines | Dan Klein, Scaled Cognition -
The Accidental Algorithm | Humans of AI Crossover with Writer's Melisa Russak -
Your Key to AI Success is Hiding in Plain Sight | Cohesity's Greg Statton -
Low-Code AI: From Requirements to Apps in Minutes | OutSystems' Rodrigo Coutinho