Prompt Engineering vs Fine Tuning vs RLHF: Full Comparison

The Three Ways to Shape LLM Behavior

When you need an LLM to do something specific, you have three main levers:

1. Prompt Engineering craft the input to guide the model

2. Fine Tuning train the model further on your data

3. RLHF align the model to human preferences via reward modeling

Each approach has a different cost, complexity, and effectiveness ceiling.

Quick Comparison Table

Prompt Engineering

What It Is

Writing system prompts, few shot examples, and instruction structures that guide the base model's output without changing its weights.

Core Techniques

Limits

• Doesn't fix underlying knowledge gaps
• Context window fills up with long examples
• Inconsistent adherence with weaker models
• Cannot teach truly novel domains

Fine Tuning

What It Is

Continuing training on a pre trained LLM using your own supervised dataset. The model's weights are updated to produce outputs matching your examples.

Types of Fine Tuning

LoRA Key Idea

# Standard layer: W is frozen
output = x @ W

# LoRA: train only A and B (small matrices)
output = x @ W + x @ (A @ B) * scale
# W shape: (d_in, d_out) — frozen
# A shape: (d_in, r), B shape: (r, d_out) — trainable
# r (rank) is typically 8, 16, or 64

When Fine Tuning Wins

• Consistent format requirements (always output valid JSON)
• Private domain knowledge not in training data
• Cost reduction (smaller model + fine tuning vs GPT-4)
• Low latency edge deployment

RLHF (Reinforcement Learning from Human Feedback)

What It Is

A three phase process that aligns a model to human preferences, used to create the instruct/chat versions of GPT, Claude, and LLaMA.

Phase 1: Supervised Fine Tuning (SFT)

Fine tune the base model on human written demonstrations.

Phase 2: Reward Model Training

Human annotators compare model outputs and rank them. A separate model is trained to predict these rankings the reward model.

Phase 3: PPO Optimization

Use the reward model as a signal for RL (Proximal Policy Optimization) to update the LLM toward higher reward outputs.

DPO (Direct Preference Optimization) Simpler Alternative

# DPO skips the explicit reward model
# Train directly on (preferred, rejected) response pairs
# Loss: maximize probability of preferred, minimize rejected
loss = -log(σ(β * log(π_θ(y_w)/π_ref(y_w)) - β * log(π_θ(y_l)/π_ref(y_l))))

DPO is now preferred over PPO RLHF for most alignment tasks simpler, stabler, and comparable quality.

Decision Framework

Need to guide an existing capable model?
  └─ Try prompt engineering first (zero cost)

Model is capable but inconsistent in format/style?
  └─ Fine-tune with LoRA (100-1K examples often enough)

Model lacks domain knowledge?
  └─ RAG for factual queries OR domain fine-tuning for reasoning

Need safety, refusal, and alignment?
  └─ RLHF/DPO at scale (requires significant resources)

Need smallest possible model for edge deployment?
  └─ Fine-tune + quantize (QLoRA → GGUF/ONNX)

Popular Tools

Common Mistakes

• Jumping to fine tuning before exhausting prompt engineering prompts are free and reversible
• Using full fine tuning when LoRA achieves the same result at 10% the compute
• Training on too few examples fine tuning on under 100 examples usually degrades generalization
• Forgetting to freeze the base model when doing LoRA (defeats the efficiency gain)
• Treating RLHF as only for safety it's a general alignment tool for any preference signal