How Large Language Models Work: A Clear Technical Explanation

How Large Language Models Work: A Clear Technical Explanation

When ChatGPT launched in November 2022, the question I heard most often from non-technical people was some version of: "How does it actually do that?"

The honest answer requires more than "it's trained on lots of text." That's like explaining how a piano sonata works by saying "the piano was pressed repeatedly." Technically true, completely uninformative.

This guide gives you the real answer — how LLMs work from first principles, through the transformer architecture, through training, through alignment. Not simplified to the point of being wrong, but explained in a way that doesn't require a PhD in machine learning.

Step 1: Text as Numbers (Tokenization)

LLMs can't process text directly — they need numbers. Tokenization converts text to sequences of integer IDs:

# Using OpenAI's tiktoken library (same tokenizer as GPT-4)
import tiktoken

enc = tiktoken.encoding_for_model("gpt-4")

text = "The transformer architecture changed everything."
tokens = enc.encode(text)
print(f"Token IDs: {tokens}")
# [791, 43678, 18112, 5614, 4395, 13]

# Decode back
decoded = [enc.decode([t]) for t in tokens]
print(f"Token strings: {decoded}")
# ['The', ' transformer', ' architecture', ' changed', ' everything', '.']

print(f"Token count: {len(tokens)}")

Modern LLMs use subword tokenization (Byte Pair Encoding or SentencePiece):

• Common words become single tokens: "the" → [791]
• Rare words split into subwords: "cryptocurrency" → ["crypto", "currency"]
• This handles any word, even invented ones, while keeping vocabulary manageable (~50K tokens)

Why this matters: The model processes tokens, not words or characters. GPT-4's cost, speed, and context window are all measured in tokens, not words.

Step 2: Embeddings (Tokens as Vectors)

Each token ID maps to a high-dimensional vector (embedding) — a point in a learned semantic space:

Vocabulary size: 50,000 tokens
Embedding dimension: 768 (BERT-base) to 12,288 (GPT-4 estimated)

Token "king"     → [0.42, -0.73, 0.15, 0.88, ...]  (768 numbers)
Token "queen"    → [0.39, -0.71, 0.18, 0.85, ...]  (similar direction)
Token "man"      → [0.11, -0.82, 0.55, 0.72, ...]
Token "woman"    → [0.08, -0.80, 0.58, 0.69, ...]

Famous example:
king - man + woman ≈ queen  (vector arithmetic captures semantic relationships)

The model learns these embeddings during training. After training, semantically related tokens have similar embedding vectors — "cat" and "feline" are close; "cat" and "spaceship" are far.

Step 3: The Transformer Architecture

The transformer is the core architectural innovation that made modern LLMs possible. It processes sequences in parallel (unlike RNNs, which process token by token) and uses attention to build context-aware representations.

Self-Attention

The key mechanism: every token "attends" to every other token in the sequence, weighting how much each other token is relevant to understanding the current one.

Input: "The bank approved the loan for the river bank"
                                           ↑
"bank" (position 8) attends to all other tokens:
- "bank" (position 2): high attention (same word, disambiguation context)
- "river" (position 7): high attention (context: this bank is a riverbank)
- "loan" (position 5): low attention (this sentence is about a different bank)
- "The" (position 1): low attention (not informative for disambiguation)

Result: the embedding for "bank" at position 8 incorporates context
        showing it means a landform, not a financial institution

Mathematically, attention computes three vectors from each token's embedding:

• Query (Q): "What am I looking for?"
• Key (K): "What do I contain?"
• Value (V): "What information do I carry?"

Attention score = softmax(Q · Kᵀ / √d) · V

Multi-Head Attention

Instead of one attention mechanism, transformers use multiple "heads" — each attending to different aspects simultaneously:

Head 1: Tracks syntactic dependencies (subject-verb agreement)
Head 2: Resolves coreference ("it" refers to "the model")
Head 3: Identifies semantic relationships (synonyms, antonyms)
Head 4-8: Other patterns the model discovered during training

Transformer Layer Stack

A complete transformer model stacks many layers, each refining representations:

Input tokens
    ↓
Token Embeddings + Positional Encoding
    ↓
[Layer 1]
  Multi-Head Self-Attention → Add & Normalize
  Feed-Forward Network → Add & Normalize
    ↓
[Layer 2]
  Multi-Head Self-Attention → Add & Normalize
  Feed-Forward Network → Add & Normalize
    ↓
[... N layers ...]
    ↓
[Layer N]
    ↓
Output head (predict next token probabilities)

GPT-3 has 96 layers. GPT-4 is estimated to have hundreds. Each layer transforms the representation — early layers capture surface features, later layers capture abstract semantics.

Step 4: Pre-Training (Learning from Text)

The pre-training objective for decoder-only models (GPT-style):

Given: "The capital of France is"
Predict: "Paris"

Given: "Paris is a city located in"
Predict: "Europe" or "France" or "western"

Training on hundreds of billions of examples like this, the model learns:
- Language patterns (grammar, syntax)
- World knowledge (facts, relationships)
- Reasoning patterns (if-then, cause-effect)
- Coding patterns (syntax, algorithms)
- And much more

The training data scale is staggering:

• GPT-3: ~300 billion tokens
• LLaMA 2: 2 trillion tokens
• GPT-4: estimated 10+ trillion tokens
• Gemini 1.5: estimated 10+ trillion tokens

One trillion tokens ≈ 750 billion words ≈ ~5 million books.

Why scale matters: With enough data and parameters, emergent behaviors appear — capabilities that weren't explicitly trained but emerge from the complexity of the learned representations. Reasoning, translation, code generation, and multi-step problem solving all emerged this way.

Step 5: Instruction Fine-Tuning

A raw pre-trained model predicts text, but it's not an assistant — it continues prompts without following instructions. Instruction fine-tuning transforms it:

Training examples for instruction fine-tuning:

[Instruction]: Summarize this article in 3 bullet points.
[Article]: [long article text]
[Response]:
• First key point...
• Second key point...
• Third key point...

[Instruction]: Write a Python function that reverses a string.
[Response]: 
def reverse_string(s):
    return s[::-1]

This teaches the model the instruction-following format: treat text between [Instruction] and [Response] as a directive, not as text to continue.

Step 6: RLHF (Reinforcement Learning from Human Feedback)

RLHF is how ChatGPT, Claude, and other assistants learn to be genuinely helpful, not just coherent:

Stage 1: Supervised Fine-Tuning (SFT)
- Human demonstrators write high-quality responses to prompts
- Model fine-tuned to mimic these responses

Stage 2: Reward Model Training
- Model generates multiple responses to the same prompt
- Human raters rank the responses (which is better?)
- A separate reward model trains on these preferences
- Reward model learns to score responses like humans would

Stage 3: PPO (Proximal Policy Optimization)
- The fine-tuned model generates responses
- Reward model scores each response
- Model is updated to generate higher-scoring responses
- Constraint: don't deviate too far from the SFT model (prevents reward hacking)

RLHF is what makes models:

• More helpful (prefer useful, actionable answers)
• Less harmful (prefer avoiding dangerous information)
• More honest (prefer admitting uncertainty to confabulation)

Why LLMs Hallucinate

LLMs generate text token by token, sampling from a probability distribution over the next token. They have no explicit fact store or truth module:

Incorrect mental model: LLM → lookup database → retrieve fact → state fact
Correct mental model:   LLM → predict what text is statistically likely 
                                given the preceding context

When you ask about a rare fact (a specific paper's authors, a small company's founded date), the model generates what would plausibly complete that sentence — based on patterns in training data, not retrieval. This produces confident-sounding hallucinations.

Mitigation:

• RAG: retrieve relevant documents before generating, ground the answer
• Tool use: let the model call APIs or databases for factual lookups
• Prompting for uncertainty: "If you're not confident, say so"
• Citations: require the model to cite the source for each claim

Key Concepts at a Glance

Conclusion

LLMs are sophisticated pattern-completion machines — not databases, not reasoning engines in the traditional sense, not systems that "understand" in the human sense. They generate text that is statistically consistent with their training data and fine-tuning, which produces remarkably capable behavior across many domains.

The architectural insight (transformers), the training approach (scale + self-supervised learning), and the alignment technique (RLHF) together explain why capabilities emerged rapidly after 2017.

For how these models compare in practical use, see our GPT-4 vs Claude vs Gemini comparison. For building applications with LLMs, see our AI development guides.

Frequently Asked Questions

What is a large language model?

A deep learning model trained on massive text data that learns to predict the next token given preceding tokens. Parameters encode learned associations between text patterns, producing emergent capabilities like reasoning, coding, and knowledge retrieval as a byproduct.

How are LLMs trained?

Three stages: pre-training on billions of tokens of text (next-token prediction, self-supervised), instruction fine-tuning on instruction-response pairs (teaches assistant behavior), and RLHF on human preference ratings (teaches alignment — helpful, harmless, honest).

Why do LLMs hallucinate?

They predict statistically likely text, not factually accurate text. No explicit truth module or fact database. When asked about rare or unknown facts, they generate plausible-sounding text that may be wrong. Mitigation: RAG, tool use, prompting for uncertainty.

What is the context window and why does it matter?

Maximum tokens the model can process at once. Ranges from 8K to 1M+ tokens depending on the model. Limits document length, conversation history, and code analysis scope. Larger context enables longer documents but increases compute cost.

What is the difference between GPT, BERT, and T5?

BERT: bidirectional, trained for understanding (fill in masked tokens). GPT: unidirectional causal, trained for generation (predict next token). T5: encoder-decoder, treats all tasks as text-to-text. Modern frontier models (GPT-4, Claude, Gemini) are decoder-only with instruction tuning and RLHF.