Attention
Builds context-sensitive relationships between token representations.
How large language models work
Here, LLM means large language model: a system trained on large collections of text to model patterns in language and generate likely continuations. This example shows the scope Nodebook should confirm before compiling an ambiguous acronym.
The core idea
A large language model takes text, splits it into smaller units called tokens, and predicts a probability distribution for what token should come next. It generates text by choosing a token, adding it to the context, and repeating the process.
From text to tokens
Models operate on numbers, not raw words. A tokenizer maps text into token identifiers. A token might be a word, part of a word, punctuation, or a short character sequence. Each token is then represented by a vector, a list of numbers that the model can transform.
Tokenisation matters because it sets the units the model sees. A familiar word may be one token while a rare technical term may be split into several pieces.
What the transformer contributes
The transformer architecture introduced a way for positions in a sequence to weigh information from other positions through attention. In simplified terms, the model can ask which earlier tokens are useful for interpreting the current one.
Feed-forward layers
Transform each position through learned nonlinear computations.
Residual pathways
Help information and gradients move through many stacked layers.
Position information
Gives the model a representation of order because attention alone does not imply sequence.
How training changes the model
During pretraining, the model repeatedly predicts tokens from examples and receives an error signal. Optimisation adjusts billions of parameters so future predictions assign more probability to continuations that match the training data.
Later stages can shape behaviour with instruction data, preference feedback, safety training, and tool use. These stages do not turn next-token prediction into human understanding, but they can make the resulting system much more useful in conversation.
Why an answer can sound right and be wrong
The model is optimised to produce plausible continuations, not to guarantee that every claim is true. It can combine patterns into a fluent statement that lacks support. Grounding a lesson in inspectable sources reduces this risk, but source quality, interpretation, and synthesis still need care.
A useful mental model is: language models are powerful pattern engines with learned representations, not databases of verified facts and not people with stable beliefs.
Try to recall it
Without looking back, explain the path from a sentence entering the system to one generated token leaving it. Your answer should mention tokenisation, vector representations, transformer layers, a probability distribution, and repeated next-token selection.