Virtual Environments & Project Setup

Virtual Environments & Project Setup: Keeping Dependencies Isolated

Virtual environments solve the "it works on my machine" problem. Without them, every Python project shares the same global packages, and updating one project's dependencies can break another. Virtual environments give each project its own isolated Python installation.

Why Virtual Environments Matter

Without virtual environments:
  Project A needs requests==2.25.0  ┐
  Project B needs requests==2.31.0  ┘ CONFLICT — only one can win
  
With virtual environments:
  Project A env: requests==2.25.0   ← Isolated
  Project B env: requests==2.31.0   ← Isolated
  System Python: no requests needed

venv: Python's Built-In Solution

# Create a virtual environment
python -m venv venv

# Activate (Windows PowerShell)
.\venv\Scripts\Activate.ps1

# Activate (Windows Command Prompt)
venv\Scripts\activate.bat

# Activate (macOS/Linux)
source venv/bin/activate

# Verify you're in the virtual environment
which python    # Should point inside venv/
pip list        # Should show only pip and setuptools

# Install packages (only affects this venv)
pip install requests pandas numpy

# Deactivate
deactivate

Project Structure: The Modern Standard

A well-structured Python project:

my-project/
├── .venv/                  ← Virtual environment (never commit this)
├── .env                    ← Environment variables (never commit this)
├── .env.example            ← Template for .env (commit this)
├── .gitignore
├── README.md
├── pyproject.toml          ← Modern project configuration
├── requirements.txt        ← Dependencies for deployment
├── requirements-dev.txt    ← Dev-only dependencies
├── src/
│   └── my_project/
│       ├── __init__.py
│       ├── main.py
│       ├── config.py
│       └── models/
├── tests/
│   ├── __init__.py
│   └── test_main.py
└── Makefile                ← Common commands

# .gitignore (always include these for Python projects)
.venv/
venv/
__pycache__/
*.pyc
*.pyo
.env
*.egg-info/
dist/
build/
.pytest_cache/
.mypy_cache/

Environment Variables with python-dotenv

# .env (never commit to git!)
DATABASE_URL=postgresql://localhost/mydb
SECRET_KEY=super-secret-key-here
DEBUG=true
API_KEY=abc123
MAX_CONNECTIONS=10

# .env.example (DO commit — shows teammates what variables are needed)
DATABASE_URL=postgresql://localhost/yourdb
SECRET_KEY=your-secret-key
DEBUG=false
API_KEY=your-api-key
MAX_CONNECTIONS=10

# config.py
from dotenv import load_dotenv
import os

load_dotenv()  # Load .env file if it exists

DATABASE_URL = os.getenv("DATABASE_URL")
SECRET_KEY = os.environ["SECRET_KEY"]  # Raises KeyError if missing
DEBUG = os.getenv("DEBUG", "false").lower() == "true"
MAX_CONNECTIONS = int(os.getenv("MAX_CONNECTIONS", "10"))

if not DATABASE_URL:
    raise ValueError("DATABASE_URL environment variable is required")

pyproject.toml: Modern Project Configuration

# pyproject.toml
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.backends.legacy:build"

[project]
name = "my-project"
version = "0.1.0"
description = "My Python project"
requires-python = ">=3.11"
dependencies = [
    "requests>=2.31.0",
    "pandas>=2.0.0",
    "pydantic>=2.0.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "black>=23.0",
    "ruff>=0.1.0",
    "mypy>=1.0",
]

[tool.black]
line-length = 88

[tool.ruff]
line-length = 88
select = ["E", "F", "W", "I"]

[tool.mypy]
python_version = "3.11"
strict = true

Poetry: The Modern Alternative

Poetry handles virtual environments, dependency resolution, and publishing:

# Install Poetry
curl -sSL https://install.python-poetry.org | python3 -

# Create new project
poetry new my-project

# Add dependencies
poetry add requests pandas

# Add dev dependencies  
poetry add --group dev pytest black ruff

# Install all dependencies
poetry install

# Run a command in the virtual environment
poetry run python main.py
poetry run pytest

# Update dependencies
poetry update

# Export to requirements.txt
poetry export -f requirements.txt --output requirements.txt

poetry.lock pins exact versions of all dependencies (including transitive dependencies) — commits this file for reproducible installs.

Managing Multiple Python Versions: pyenv

Different projects may need different Python versions:

# Install pyenv (macOS/Linux)
curl https://pyenv.run | bash

# List available Python versions
pyenv install --list

# Install a specific version
pyenv install 3.11.7
pyenv install 3.12.0

# Set global default
pyenv global 3.11.7

# Set version for a specific directory
cd my-project
pyenv local 3.12.0   # Creates .python-version file

# Check current Python
pyenv version

# Windows alternative: py launcher
py -3.11 script.py
py -3.12 -m venv .venv

Common Makefile Commands

# Makefile — run commands with 'make <target>'
.PHONY: install test lint format clean

install:
	python -m venv .venv
	.venv/bin/pip install -e ".[dev]"

test:
	.venv/bin/pytest tests/ -v

lint:
	.venv/bin/ruff check src/
	.venv/bin/mypy src/

format:
	.venv/bin/black src/ tests/
	.venv/bin/ruff check --fix src/

clean:
	find . -type d -name __pycache__ -exec rm -rf {} +
	find . -type f -name "*.pyc" -delete
	rm -rf .pytest_cache .mypy_cache dist build *.egg-info

Docker: The Ultimate Isolation

For production deployments, Docker provides OS-level isolation:

# Dockerfile
FROM python:3.11-slim

WORKDIR /app

# Install dependencies first (cached layer)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy source code
COPY src/ ./src/

# Run the application
CMD ["python", "-m", "src.main"]

# Build and run
docker build -t my-project .
docker run --env-file .env -p 8000:8000 my-project

The Complete Setup Checklist

Starting a new Python project:

# 1. Create project directory
mkdir my-project && cd my-project
git init

# 2. Create virtual environment
python -m venv .venv
.venv\Scripts\Activate.ps1  # Windows
# source .venv/bin/activate  # Linux/Mac

# 3. Create .gitignore
echo ".venv/\n__pycache__/\n.env\n*.pyc" > .gitignore

# 4. Create .env and .env.example
touch .env .env.example

# 5. Install initial dependencies
pip install requests python-dotenv

# 6. Save dependencies
pip freeze > requirements.txt

# 7. Create project structure
mkdir -p src tests
touch src/__init__.py tests/__init__.py

# 8. First commit
git add .gitignore requirements.txt src/ tests/
git commit -m "Initial project setup"

Every serious Python developer works with virtual environments. Setting up the right project structure from the start saves hours of configuration problems later.

Next lesson: Generators & Iterators — Python's powerful tools for memory-efficient iteration.