AI Engineer Roadmap: Skills, Tools and Free Resources

AI Engineer Roadmap: Skills, Tools and Free Resources

An AI engineer builds software products on top of models that already exist — retrieval systems, agents, evaluation harnesses and the unglamorous backend plumbing that keeps them fast, cheap and correct. Expect twelve to twenty-four months from zero, or six to twelve if you already ship production software.

Updated for 2026. Salary figures are indicative and move quarterly.

What This Role Actually Does

Most of the job is normal software engineering with an unreliable component in the middle.

A realistic week looks like this. Two days on retrieval quality, because the model answered incorrectly and the cause was that the right document never reached it. One day on evaluation, building a test set of real user questions and scoring output changes against it. One day on cost and latency, because a feature that works costs eleven cents per request and needs to cost two. One day on the ordinary work of any engineer — logging, error handling, deployment, a broken CI pipeline.

What people wrongly imagine it is: training models, tuning neural networks, reading research papers all day, and writing brilliant prompts. In reality, model training belongs to the machine learning engineer, and prompt writing is perhaps three percent of the work.

The uncomfortable truth is that AI engineering is mostly backend engineering — with a probabilistic dependency you cannot fully control and therefore must measure.

The Roadmap

Six stages. Do not skip stage one, however tempting the interesting parts look.

Stage 1 — Programming Foundations in Python

What to learn: Python syntax, data structures, functions, classes, virtual environments, pip, error handling, file and JSON handling, and reading a stack trace without panic. Add Git and the command line.

Realistic time: 2–3 months from zero. 2 weeks if you already program in another language.

Free resources: the official Python Tutorial on python.org, freeCodeCamp's Scientific Computing with Python certification, and Automate the Boring Stuff with Python (free to read online). For Git, the free Pro Git book.

Portfolio project: a command-line tool that reads a folder of text files, extracts something structured from each one, and writes a CSV. Boring on purpose — it proves file handling, error handling and iteration.

You are done when: you can write a 150-line script from a blank file without copying structure from a tutorial, and you can explain what a virtual environment is for. Keep the Python cheat sheet open while you work.

Stage 2 — Backend Engineering and APIs

What to learn: HTTP, REST, JSON, authentication basics, environment variables and secrets, FastAPI or Flask, PostgreSQL and enough SQL to model data properly, async programming in Python, and Docker basics.

Realistic time: 3–4 months.

Free resources: the FastAPI official documentation, which is unusually good as a teaching text; MDN Web Docs for HTTP fundamentals; PostgreSQL Tutorial and SQLBolt for SQL; and the official Docker getting-started guide.

Portfolio project: a deployed REST API with authentication, a database, input validation and a public URL. No AI in it yet. This is the project that proves you are an engineer.

You are done when: your API is live on the internet, survives bad input without a 500 error, and you can explain how a request travels from client to database and back. The SQL cheat sheet is worth keeping to hand.

Stage 3 — Working With Language Models

What to learn: tokens and context windows, temperature and sampling, structured output and function calling, streaming responses, rate limits, retries and backoff, cost per token, and the practical failure modes — hallucination, truncation, refusal, latency spikes.

Realistic time: 1–2 months.

Free resources: the Anthropic and OpenAI official API documentation and prompt engineering guides, the free DeepLearning.AI short courses on building with large language models, and the Hugging Face LLM course.

Portfolio project: a document question-answering service, built directly against a model API with no framework. Writing it without a framework first is the point — you learn what the frameworks are hiding.

You are done when: you can estimate the token cost of a feature before building it, and you handle a rate-limit error correctly rather than crashing.

Stage 4 — Retrieval and Embeddings

What to learn: embeddings and vector similarity, chunking strategies and why they dominate output quality, vector databases such as pgvector, Qdrant or Chroma, hybrid search combining keyword and vector retrieval, reranking, and the full retrieval-augmented generation pattern.

Realistic time: 2–3 months.

Free resources: the Hugging Face NLP course sections on embeddings, the official pgvector and Qdrant documentation, and DeepLearning.AI's short courses on retrieval-augmented generation and vector databases.

Portfolio project: a retrieval system over a real corpus you care about — your company's docs, a textbook, a legal code — with a written comparison of at least two chunking strategies and the measured difference between them.

You are done when: you can diagnose a wrong answer as a retrieval failure versus a generation failure, and you can prove which one it was.

Stage 5 — Evaluation, Guardrails and Cost Control

What to learn: building golden test sets, automated scoring including model-graded evaluation, regression testing for prompts, tracing and observability, caching strategies, input and output validation, and safe handling of untrusted user input reaching a model.

Realistic time: 2–3 months. This is the stage that separates hobby projects from employable ones.

Free resources: the OpenAI Evals repository, Ragas documentation for retrieval evaluation, LangSmith and Langfuse free tiers and their documentation, and the OWASP Top 10 for Large Language Model Applications.

Portfolio project: take your stage four project and add an evaluation suite of fifty real questions with expected answers, a scoring script, and a written report showing how three changes affected the score.

You are done when: you can say "this change improved accuracy from 71 to 78 percent on our fifty-question set" instead of "this prompt feels better".

Stage 6 — Agents, Deployment and Production

What to learn: tool use and function calling at scale, multi-step agent loops and when not to use them, background jobs and queues, streaming to a frontend, monitoring and alerting, deployment on a cloud provider, and cost dashboards.

Realistic time: 2–4 months.

Free resources: the Model Context Protocol documentation for tool integration, DeepLearning.AI agent short courses, and the free tiers plus official docs of AWS, Google Cloud or Cloudflare Workers for deployment.

Portfolio project: one production-grade application, deployed publicly, with authentication, retrieval, evaluation, logging, a cost ceiling, and a README documenting one thing that failed and how you fixed it.

You are done when: a stranger can use your application without you present, and you can answer what it costs per thousand requests.

Salary and Job Titles

Titles you will see: AI Engineer, LLM Engineer, Applied AI Engineer, AI Software Engineer, Machine Learning Engineer (Applied), and Forward Deployed Engineer at some AI companies.

These are aggregate indicative ranges drawn from the kind of self-reported and employer-reported data published by Levels.fyi, Glassdoor and the Stack Overflow Developer Survey, plus occupational data from the US Bureau of Labor Statistics for the broader software developer category. Equity is excluded and can be substantial at large employers.

State it plainly: these figures are indicative, vary enormously by employer and city, and move quarterly. Use the gaps between rows, not the absolute numbers.

Who Should Not Take This Path

Be honest with yourself before spending two years here.

Do not take this path if you want deterministic systems. The core frustration of this role is that the same input can produce different output. If a flaky test ruins your week, a flaky model will ruin your year.

Do not take this path if you dislike measurement. Most of the real work is designing evaluations and arguing about whether an improvement is real. People who want to build and move on find this tedious.

Do not take this path if you want to avoid backend engineering. There is no version of this job that skips APIs, databases, deployment and debugging. If that sounds like a chore rather than a foundation, data analysis or UI/UX design will suit you far better.

Do not take this path if you wanted to do research. Publishing, novel architectures and training runs belong to research scientists and, partly, to machine learning engineers. This role consumes research, it does not produce it.

Do not take this path if tool churn upsets you. The framework you learn this quarter may be unfashionable next year. The fundamentals last; the libraries do not.

The Five Mistakes

1. Starting at stage three. Building chatbots before you can build an API produces someone who cannot be hired, because every interview loop includes ordinary engineering questions. The model layer is the thin part.

2. Learning a framework instead of the concepts. People who learn only an orchestration library cannot debug it when it misbehaves. Build one retrieval system from raw API calls first, then adopt a framework knowing exactly what it saves you.

3. Never measuring anything. "The output looks better" is not an engineering claim. Candidates who bring a real evaluation table to an interview immediately outrank candidates with prettier demos.

4. Ignoring cost and latency. A feature that is correct and takes eleven seconds is a feature that ships disabled. Track tokens, cache aggressively, and know your per-request cost before anyone asks.

5. Waiting for a first job with this exact title. Most AI engineers arrived sideways from software or data roles. Take the adjacent job, own the model integration work, and move internally.

Print This Section

AI ENGINEER ROADMAP - STAGE SUMMARY

1. PYTHON FOUNDATIONS                       2-3 months
   Python, Git, CLI, JSON, error handling
   Project: CLI tool that extracts data to CSV
   Done when: 150-line script from blank file

2. BACKEND ENGINEERING                      3-4 months
   HTTP, FastAPI, PostgreSQL, SQL, Docker, auth
   Project: deployed REST API with a database
   Done when: it is live and survives bad input

3. WORKING WITH MODELS                      1-2 months
   Tokens, context, structured output, retries, cost
   Project: doc Q&A with no framework
   Done when: you can estimate cost before building

4. RETRIEVAL AND EMBEDDINGS                 2-3 months
   Embeddings, chunking, pgvector/Qdrant, hybrid search
   Project: RAG over a real corpus + chunking comparison
   Done when: you can separate retrieval vs generation failure

5. EVALUATION AND GUARDRAILS                2-3 months
   Golden sets, scoring, tracing, caching, OWASP LLM Top 10
   Project: 50-question eval suite + measured report
   Done when: you quote numbers, not feelings

6. AGENTS AND PRODUCTION                    2-4 months
   Tool use, queues, streaming, monitoring, deployment
   Project: one production app with auth, logging, cost cap
   Done when: a stranger can use it without you

TOTAL: 12-24 months from zero, 6-12 if already an engineer
US entry 110k-160k USD | Canada entry 95k-130k CAD
Figures are indicative and move quarterly.

🔗 Read next: the machine learning engineer learning path if you want to train models rather than build on them, or go back to the pillar — every tech career roadmap compared.