Prompt Engineering vs Fine-Tuning vs RLHF: Full Comparison

The Three Ways to Shape LLM Behavior

When you need an LLM to do something specific, you have three main levers:

1. Prompt Engineering — craft the input to guide the model

2. Fine-Tuning — train the model further on your data

3. RLHF — align the model to human preferences via reward modeling

Each approach has a different cost, complexity, and effectiveness ceiling.

Quick Comparison Table

Prompt Engineering

What It Is

Writing system prompts, few-shot examples, and instruction structures that guide the base model's output without changing its weights.

Core Techniques

Limits

• Doesn't fix underlying knowledge gaps
• Context window fills up with long examples
• Inconsistent adherence with weaker models
• Cannot teach truly novel domains

Fine-Tuning

What It Is

Continuing training on a pre-trained LLM using your own supervised dataset. The model's weights are updated to produce outputs matching your examples.

Types of Fine-Tuning

LoRA Key Idea

# Standard layer: W is frozen
output = x @ W

# LoRA: train only A and B (small matrices)
output = x @ W + x @ (A @ B) * scale
# W shape: (d_in, d_out) — frozen
# A shape: (d_in, r), B shape: (r, d_out) — trainable
# r (rank) is typically 8, 16, or 64

When Fine-Tuning Wins

• Consistent format requirements (always output valid JSON)
• Private domain knowledge not in training data
• Cost reduction (smaller model + fine-tuning vs GPT-4)
• Low-latency edge deployment

RLHF (Reinforcement Learning from Human Feedback)

What It Is

A three-phase process that aligns a model to human preferences, used to create the instruct/chat versions of GPT, Claude, and LLaMA.

Phase 1: Supervised Fine-Tuning (SFT)

Fine-tune the base model on human-written demonstrations.

Phase 2: Reward Model Training

Human annotators compare model outputs and rank them. A separate model is trained to predict these rankings — the reward model.

Phase 3: PPO Optimization

Use the reward model as a signal for RL (Proximal Policy Optimization) to update the LLM toward higher-reward outputs.

DPO (Direct Preference Optimization) — Simpler Alternative

# DPO skips the explicit reward model
# Train directly on (preferred, rejected) response pairs
# Loss: maximize probability of preferred, minimize rejected
loss = -log(σ(β * log(π_θ(y_w)/π_ref(y_w)) - β * log(π_θ(y_l)/π_ref(y_l))))

DPO is now preferred over PPO-RLHF for most alignment tasks — simpler, stabler, and comparable quality.

Decision Framework

Need to guide an existing capable model?
  └─ Try prompt engineering first (zero cost)

Model is capable but inconsistent in format/style?
  └─ Fine-tune with LoRA (100-1K examples often enough)

Model lacks domain knowledge?
  └─ RAG for factual queries OR domain fine-tuning for reasoning

Need safety, refusal, and alignment?
  └─ RLHF/DPO at scale (requires significant resources)

Need smallest possible model for edge deployment?
  └─ Fine-tune + quantize (QLoRA → GGUF/ONNX)

Popular Tools

Common Mistakes

• Jumping to fine-tuning before exhausting prompt engineering — prompts are free and reversible
• Using full fine-tuning when LoRA achieves the same result at 10% the compute
• Training on too few examples — fine-tuning on under 100 examples usually degrades generalization
• Forgetting to freeze the base model when doing LoRA (defeats the efficiency gain)
• Treating RLHF as only for safety — it's a general alignment tool for any preference signal