Best Open Source LLMs in 2025: LLaMA, Mistral, Phi and More Compared

Best Open Source LLMs in 2025: LLaMA, Mistral, Phi and More Compared

When Meta released LLaMA 2 in 2023, I downloaded it expecting a novelty. Instead, I got a model that was legitimately useful for summarization and analysis tasks — running entirely on my laptop, with no API costs and no data leaving my machine.

By 2025, the open-source LLM ecosystem has transformed. LLaMA 3.1 70B competes with GPT-3.5-Turbo on most benchmarks. Phi-3 Mini achieves GPT-3.5-level performance at 3.8 billion parameters. And the infrastructure to run these models locally — Ollama, llama.cpp, vLLM — has made deployment trivially easy.

This guide covers the models worth using in 2025, their hardware requirements, and when open-source makes more sense than paying for API access.

The Landscape: Open-Source Model Families

Meta (LLaMA family):
- LLaMA 3.1 8B, 70B, 405B
- Code LLaMA 7B, 13B, 34B, 70B
- LLaMA 3.2 Multimodal (1B, 3B, 11B, 90B)

Mistral AI:
- Mistral 7B v0.3
- Mixtral 8x7B (MoE — 46.7B total, 12.9B active)
- Mixtral 8x22B (MoE — 141B total, 39B active)

Microsoft:
- Phi-3 Mini 3.8B, Small 7B, Medium 14B
- Phi-3.5 (improved versions)

Google:
- Gemma 2B, 7B
- Gemma 2 9B, 27B (significantly better)
- CodeGemma

Alibaba:
- Qwen2 0.5B, 1.5B, 7B, 72B, 110B
- Qwen2.5-Coder (excellent for coding)

DeepSeek:
- DeepSeek V2 (strong general model)
- DeepSeek-Coder V2 (top open-source coding)

Model Comparison: 2025

Hardware Requirements

Quick Reference

Model Size → Minimum VRAM (4-bit quantization):

~7B models (Mistral, LLaMA 3.1 8B, Phi-3 Small):
  - 6-8 GB VRAM (RTX 3060, RTX 4060)
  - Or 8-16 GB RAM for CPU inference (slow)

~13-14B models (Phi-3 Medium, Code LLaMA 13B):
  - 10-12 GB VRAM (RTX 3080, RTX 4070)
  - Or 32 GB RAM for CPU

~34-40B models (Code LLaMA 34B, Mixtral 8x7B):
  - 24-28 GB VRAM (RTX 3090/4090 or 2× RTX 3080)
  - Or 64 GB RAM for CPU

~70B models (LLaMA 3.1 70B):
  - 40-48 GB VRAM (2× A100 40GB, or 2× RTX 3090)
  - Or 128 GB RAM for CPU (very slow)

~405B (LLaMA 3.1 405B):
  - 8× A100 80GB (not consumer-feasible)
  - Use via API (Together AI, Groq)

Running Models Locally with Ollama

Ollama is the easiest way to run open-source models:

# Install Ollama (Mac/Linux/Windows)
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run a model — one command
ollama run llama3.1

# Or specify size
ollama run llama3.1:70b

# Other popular models
ollama run mistral
ollama run phi3
ollama run codellama
ollama run gemma2

# List downloaded models
ollama list

# Remove a model
ollama rm mistral

Using Ollama's OpenAI-Compatible API

from openai import OpenAI

# Ollama exposes OpenAI-compatible endpoint
client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # Not required but must be set
)

response = client.chat.completions.create(
    model="llama3.1",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain gradient descent in simple terms."}
    ],
    temperature=0.7
)

print(response.choices[0].message.content)

# Streaming response
stream = client.chat.completions.create(
    model="llama3.1",
    messages=[{"role": "user", "content": "Write a Python function to sort a list"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

High-Performance Inference with vLLM

For production deployments requiring high throughput:

# Install vLLM
# pip install vllm

from vllm import LLM, SamplingParams

# Load model (downloads from HuggingFace on first run)
llm = LLM(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    max_model_len=4096,    # Max context length
    gpu_memory_utilization=0.9,  # Use 90% of GPU memory
    quantization="awq",    # Use AWQ quantization for memory efficiency
)

# Define sampling parameters
sampling_params = SamplingParams(
    temperature=0.7,
    top_p=0.95,
    max_tokens=512
)

# Batch inference (vLLM's strength: continuous batching)
prompts = [
    "Explain machine learning to a 10-year-old.",
    "What are the main differences between Python and JavaScript?",
    "Write a haiku about programming.",
]

outputs = llm.generate(prompts, sampling_params)

for output in outputs:
    print(f"Prompt: {output.prompt}")
    print(f"Response: {output.outputs[0].text}\n")

vLLM OpenAI-Compatible Server

# Start vLLM server (drop-in replacement for OpenAI API)
python -m vllm.entrypoints.openai.api_server \
    --model meta-llama/Meta-Llama-3.1-8B-Instruct \
    --host 0.0.0.0 \
    --port 8000 \
    --quantization awq

# Now query it with standard OpenAI SDK
# Just change base_url to http://localhost:8000/v1

Model Deep-Dives

LLaMA 3.1: The New Baseline

Meta's LLaMA 3.1 is the benchmark for open-source models:

# Fine-tuning LLaMA 3.1 8B with Unsloth (fast, memory efficient)
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",
    max_seq_length=2048,
    dtype=None,  # Auto-detect
    load_in_4bit=True,
)

# Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,               # LoRA rank
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",  # Memory optimization
)

# LLaMA 3.1 chat format
def format_llama3_chat(system: str, user: str) -> str:
    return f"""<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system}<|eot_id|><|start_header_id|>user<|end_header_id|>
{user}<|eot_id|><|start_header_id|>assistant<|end_header_id|>"""

Mistral 7B: Best Apache 2.0 Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "mistralai/Mistral-7B-Instruct-v0.3"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,  # Half precision
    device_map="auto"
)

# Mistral instruction format
messages = [
    {"role": "user", "content": "What is the difference between RAG and fine-tuning?"}
]

# Apply chat template
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt").to("cuda")
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True
)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Phi-3: Small but Mighty

# Phi-3 Mini runs in Google Colab free tier
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="microsoft/Phi-3-mini-4k-instruct",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "Write a Python function to find prime numbers up to n"}
]

output = pipe(messages, max_new_tokens=300, temperature=0.3)
print(output[0]["generated_text"][-1]["content"])

Choosing the Right Open-Source Model

Hosted Open-Source APIs (Best of Both Worlds)

If you want open-source quality without managing hardware:

from openai import OpenAI

# Together AI — hosted open-source models, OpenAI-compatible
together_client = OpenAI(
    base_url="https://api.together.xyz/v1",
    api_key="your-together-api-key"
)

# ~10-30x cheaper than GPT-4 for comparable quality models
response = together_client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Explain transformer architecture"}]
)

# Groq — ultra-fast inference (LPU hardware), free tier available
groq_client = OpenAI(
    base_url="https://api.groq.com/openai/v1",
    api_key="your-groq-api-key"
)

# 300+ tokens/second — much faster than OpenAI
fast_response = groq_client.chat.completions.create(
    model="llama-3.1-70b-versatile",
    messages=[{"role": "user", "content": "What is LLaMA?"}]
)

Conclusion

The open-source LLM ecosystem in 2025 is genuinely competitive with closed-source alternatives for most use cases. LLaMA 3.1 70B competes with GPT-3.5-Turbo. Mistral 7B handles most business tasks at a fraction of the API cost. Phi-3 Mini runs on a phone.

The decision is now about tradeoffs — not capability. Choose open-source when privacy, cost at scale, or customization matter. Choose closed-source APIs when you need maximum quality with minimal engineering overhead.

For running LLMs locally with a friendly interface, see our Ollama tutorial. For understanding how to make these models even better with fine-tuning, see our fine-tuning LLM guide.

Frequently Asked Questions

What are the best open source LLMs in 2025?

LLaMA 3.1 70B for near-frontier quality; Mistral 7B for Apache 2.0 commercial use; Phi-3 Mini for consumer hardware; DeepSeek Coder V2 for coding; Qwen2 72B for multilingual tasks. The gap between open and closed source has closed dramatically — LLaMA 3.1 70B beats GPT-3.5 on most benchmarks.

Can I run LLaMA 3 locally on my computer?

Yes. LLaMA 3.1 8B (4-bit quantized) runs on 8GB VRAM or 16GB RAM. For 70B you need ~40GB VRAM. Ollama makes setup trivial — one command downloads and runs any model. Quantization (GGUF format) enables consumer hardware inference.

What is the difference between base and instruct models?

Base models predict next tokens from pre-training — not conversation-ready. Instruct models are fine-tuned to follow instructions and have conversations. Always use instruct variants for practical tasks. Base models are for further fine-tuning.

What licenses do open source LLMs use?

Apache 2.0 (Mistral, Phi-3, Gemma) allows free commercial use. LLaMA 3.1 has its own permissive license, free for most commercial use. Always read license terms before commercial deployment — Mistral (Apache 2.0) is safest.

How do I choose between local models vs API?

Local: when privacy matters, at scale cost reduction, air-gapped environments, fine-tuning on proprietary data. API: when hardware isn't available, maximum quality needed, minimal ops overhead. Middle ground: hosted open-source APIs (Together AI, Groq) give open-source models without managing hardware.