What a Feed-Forward Network Does Inside a Transformer • didof.dev

Attention gets almost all the fame. It finds which earlier words matter, moves information between tokens, and produces the diagrams with the colorful lines.

Then attention ends, and the Transformer block is still not finished.

Each token now carries a representation shaped by its context. The representation for “bank” can differ between “river bank” and “investment bank.” Yet collecting the relevant context is not the same as turning it into a useful update. Another large sublayer does that work: the feed-forward network, or FFN.

The clean mental model is this: attention mixes information across token positions; the FFN transforms each resulting position independently.

That sentence is useful, but it is not a border wall. Attention also contains learned projections, and the whole stack contributes to the model’s computation. Still, the distinction exposes the job that disappears when every Transformer explanation stops at attention.

What leaves attention?

A Transformer does not pass words between its layers. It passes vectors, lists of numbers with a fixed length called the model dimension.

Before attention, the vector at one position contains the model’s current representation of that token. Attention compares positions and combines projected information from the positions that matter. Its output at each position is another vector of the same model width, now changed by context.

So “contextualized token” does not mean that the token has acquired a paragraph of readable notes. It means that its numbers now depend on other allowed positions in the sequence.

The FFN receives one of those vectors at a time. Inside this sublayer, the token at position 12 does not read position 11 or position 13. That cross-position exchange already happened in attention. The FFN applies the same learned function separately to every position.

Follow one token through the FFN

The original Transformer gives us one useful dimensional example. Its model vectors contain 512 numbers. Its FFN expands each vector to 2,048 intermediate values, applies a nonlinearity, then contracts the result back to 512.

contextualized token x
        512
         |
         | first learned projection
         v
       2,048
         |
         | ReLU activation
         v
       2,048
         |
         | second learned projection
         v
   512-value update

In the original paper’s equation, the operation is:

FFN(x) = ReLU(x W1 + b1) W2 + b2

The first matrix, W1, computes 2,048 learned combinations of the 512 input values. Expansion does not give the token more context. No other position enters the calculation. It gives the network a wider intermediate space in which many input-dependent signals can be represented at once.

The rectified linear unit, or ReLU, keeps positive intermediate values and replaces negative ones with zero. This small nonlinear step matters. Without it, two consecutive linear projections could be collapsed into one linear projection. The expansion and contraction would have less expressive effect.

The second matrix, W2, combines the surviving intermediate signals into a 512-value update. The vector returns to the model dimension so the block can add that update to the stream that continues through the network.

The intermediate values are not 2,048 labeled drawers such as “river,” “finance,” or “plural noun.” Some may correlate with patterns that humans can interpret, but the computation is distributed and learned. The dimensions tell us the shape of the operation, not a dictionary for reading its mind.

Why the same FFN treats tokens differently

Every position in one layer uses the same W1, W2, and biases. “Applied separately and identically” can therefore sound as if the FFN stamps the same result onto every token.

It does not, because the function is shared but the inputs are not.

Attention has already made the vector for “bank” depend on its sentence. Different input numbers produce different intermediate values. ReLU then keeps a different pattern of positive values for each input, and W2 combines that pattern into a different update.

This is similar to applying the same image filter to every pixel neighborhood: the rule stays fixed while the output follows the local input. The analogy stops at locality. An FFN works on a learned token representation, not on visible colors or a hand-designed kernel.

There is one precise limit to the claim. If two positions enter a deterministic FFN with exactly the same vector, the FFN produces exactly the same output for both. Position-wise processing can respond to contextual differences. It cannot invent a distinction that is absent from its input.

Why the FFN contains so many parameters

The dimensional path also explains the weight count.

In the original Transformer, the expansion matrix has 512 x 2,048 = 1,048,576 weights. The contraction matrix has the same number in the opposite shape. Ignoring the much smaller bias vectors, one FFN therefore contains:

1,048,576 + 1,048,576 = 2,097,152 weights

The self-attention sublayer in the same encoder block learns projections for queries, keys, values, and the combined output. Across all heads, those four projections contain the equivalent of four 512 x 512 matrices:

4 x 512 x 512 = 1,048,576 weights

On that accounting boundary, the FFN has roughly twice as many projection weights as self-attention. The paper title said attention was all you need. The parameter ledger was less theatrical.

This ratio is not a law of Transformers. Gated FFNs add another input projection. Grouped-query attention reduces some attention projections. Multi-head latent attention uses a different parameterization. Decoder blocks may also contain cross-attention. Dimensions, biases, parameter sharing, and low-rank structure all move the result.

The safe conclusion is narrower: expanding a model-width vector into a much wider learned space, at every layer, requires large matrices. FFNs often own a substantial fraction of a Transformer’s parameters for exactly that reason.

The residual path does not erase the input

The 512-value FFN output is usually not asked to replace the token representation on its own. The block adds it to a residual path.

In simplified form:

y = x + FFN(normalize(x))

The original Transformer placed normalization after the addition, while many later models normalize before the sublayer. The ordering changes, but the additive route remains the key idea.

The FFN can learn an update instead of reconstructing the entire representation from scratch. If its update is small, the input has a direct path forward. Matching input and output widths makes that addition possible and lets many blocks refine a shared residual stream.

It is tempting to say the residual connection “preserves the information.” That is too strong. Addition can reinforce, cancel, or rotate features once later layers act on the result. What the architecture preserves is a direct computational route around the sublayer, not a guarantee that every semantic detail remains untouched.

Modern FFNs often use a gate

ReLU is easy to see, but many modern language models use a gated FFN. SwiGLU, a variant of the Gated Linear Unit, is a common example. Noam Shazeer’s SwiGLU paper replaces the single expanded branch with two learned projections of the same input.

One branch passes through the smooth Swish activation. Its values are multiplied element by element with the second branch, then the result is projected back to the model width.

                 -> Swish(up(x)) --
x -> two branches                  multiply -> down -> update
                 -> gate(x) ------

Calling the second branch a gate is helpful as long as we do not imagine a row of hard switches. Its learned continuous values can reduce, amplify, or reverse contributions from the activated branch.

Llama 3 uses SwiGLU, but that does not make SwiGLU universal. Other models use Gaussian Error Linear Units, ReLU variants, different gates, or different channel-mixing modules. The stable part of the mental model is the wide, input-dependent transformation followed by a projection back to the residual width.

Why Mixture of Experts duplicates the FFN

That makes the architectural choice behind Mixture of Experts, or MoE, easier to see.

The FFN is expensive because it contains large matrices. It is also token-local because one token can pass through it without reading the other positions. That makes it a natural place for conditional computation.

Instead of giving a layer one FFN, a sparse MoE model stores several. A router looks at each token’s current representation and selects a small subset of those alternative transformations. Attention and the rest of the block can remain shared.

DeepSeek-V3 makes the substitution visible in its official inference implementation: a block chooses either a dense multilayer perceptron or an MoE module for its FFN, and each expert uses a gated transformation. The full routing story, including why inactive experts still cost memory and communication, is in the companion Mixture of Experts walkthrough.

MoE does not duplicate a neat shelf of factual databases. It duplicates parameterized ways to transform a token. Routing decides which transformations run for that token at that layer.

Is the FFN the model’s memory?

There is real evidence behind the memory language. Mor Geva and colleagues analyzed Transformer FFNs as key-value memories: intermediate patterns behaved like learned keys, while output vectors contributed predictions associated with those patterns.

But “memory” is an interpretation of learned behavior, not the declared data structure of an FFN.

An FFN does not perform an exact lookup in a table with one row per fact. Many intermediate values can activate together. One value can participate in many contexts. Outputs accumulate through residual connections and later layers. Attention determines which contextual information reaches the transformation in the first place.

So replacing “the FFN is plumbing” with “the FFN is the database” only trades one bad shortcut for another.

The better model is a repeated division of labor. Attention lets token positions communicate. The FFN changes each contextualized representation through a wide, nonlinear transformation. The residual stream carries the input and the update onward, where the next block can do both again.

Attention did not finish the thought. It prepared the state that the rest of the block could transform.