From Token to Transformer - 07 / 10
Query, Key, and Value are often compared to a search request, an index, and a document. That analogy is a useful start, but it leaves an important question: why does the same token need three representations?
Attention has two distinct jobs. It must decide which positions should connect, and then decide what information should pass through those connections. Q, K, and V let the model learn matching and content transfer in separate spaces.
Three projections from the same state
In self-attention, let the input matrix be X. The model creates:
textQ = XW_Q K = XW_K V = XW_V
The matrices W_Q, W_K, and W_V are trained with the rest of the network. Q, K, and V are not fields created by the tokenizer and they do not use separate vocabularies. They are projections of the same hidden states.
The same token therefore receives different Q, K, and V vectors in different layers, heads, and contexts.
Query: features used to request information
A Query contains the features a position uses for matching. It does not have to correspond to a question that can be translated into English.
In a pronoun position, training may produce a Query that matches plausible entity representations. At a variable reference in code, a Query may match visible definitions. These are observed tendencies, not roles assigned in advance.
Key: features used to be found
Every source position offers a Key. A Query-Key dot product measures compatibility in the learned projection space.
The Key is not the complete content of a token. It is a searchable representation. A position can expose features that help answer "am I the source you need?" while carrying a different set of features in its Value.
That separation gives the model room to make something easy to identify without forcing the retrieved content to use the same coordinates.
Value: information passed through the match
Attention weights are applied to Values, not Keys. Once the model has computed the ranking, it forms a weighted mixture of the content representations.
The full path is:
textscores = QK^T / sqrt(d_k) weights = softmax(scores + mask) output = weights V
Matching, structural constraints, and information transfer are explicit stages.
Why divide by the square root of the dimension
As vector width increases, raw dot products tend to grow in magnitude. Large logits can make softmax extremely sharp and gradients harder to optimize.
Dividing by sqrt(d_k) corrects the scale so that scores remain in a more stable range. It is an optimization detail with architectural consequences, not a semantic change to the ranking.
The projections do not form a fixed role table
It is tempting to label a head as "the syntax head" or "the coreference head." Some heads do show repeatable behavior, but a head can serve different purposes across inputs, overlap with others, or be removed with little effect.
A safer mental model is that Q, K, and V define a general mechanism in which the network learns a matching space and a content space. Interpretability work can study the patterns that emerge; the architecture does not preassign a human-readable meaning to each dimension.
Cross-attention changes the source
In self-attention, all three projections originate from one sequence. In cross-attention, the Query usually comes from the representation being updated, while Key and Value come from a different source.
In translation, a decoder Query searches encoder Keys and retrieves encoder Values. This arrangement directly connects "what the decoder needs now" with "what the source sentence can provide."
Takeaway
Q, K, and V separate three responsibilities:
- Query expresses the features used to initiate a match.
- Key expresses the features used to receive that match.
- Value carries the information transferred after matching.
One set of projections is enough to run attention. The next article explains why Transformers run several sets in parallel through multi-head attention.