Neural Networks, Actually Explained

A neural network is built out of tiny units called neurons. Each neuron is simple. It takes in a few numbers, multiplies each by a weight, adds them up, and runs the total through a simple squishing function. That is it.

One neuron, step by step

1. 1Receive inputs: x1, x2, x3
2. 2Multiply by weights: w1·x1 + w2·x2 + w3·x3
3. 3Add a bias number: total + b
4. 4Pass through an activation function like ReLU or sigmoid
5. 5Send the result to the next neuron

Stacking into layers

A layer is many neurons working at the same time, each with its own weights. Stack many layers in a row and you have a deep neural network. The output of one layer becomes the input to the next.

# A tiny feed-forward network in pseudo-code
h1 = relu(W1 @ x + b1)
h2 = relu(W2 @ h1 + b2)
output = softmax(W3 @ h2 + b3)

What gets learned

• The weights W and biases b in every layer
• Nothing else — the structure stays fixed during training
• A modern LLM has hundreds of billions of these numbers

Why deep beats shallow

Early layers pick up simple features like edges in images or letter patterns in text. Middle layers combine those into shapes or word parts. Late layers pull it all together into concepts like dog or sarcasm. Depth lets the model build up abstractions.

“The network's layers are a ladder from pixels to meaning.”

The big idea: a neural network is a huge stack of very simple math units, wired together in layers. Depth and scale turn that simplicity into something that can translate languages and write poetry.