5 CI/CD Pipeline Best Practices (GitHub Actions and GitLab CI)

I've set up CI/CD pipelines for teams that ranged from three developers to three hundred. The failures I've seen are almost never about the tool. They're about choices made in the first week that seemed reasonable and caused pain for the next two years.

These five practices aren't theoretical. Each one came from watching a pipeline cause production problems and then fixing the root cause.

According to the 2023 DORA State of DevOps Report, elite performing teams deploy 182x more frequently than low performers and recover from incidents 2,604x faster. The difference is almost always automation quality — not team size or budget.

What We're Building Toward

Before the practices, here's a target pipeline structure. Most applications should aim for something like this:

push/PR → lint → test → build → security scan → deploy (staging) → deploy (prod)

Each stage only runs if the previous one passes. Deployment to production requires explicit approval or a specific trigger (merge to main). Simple in theory; the details are where things break.

Practice 1: Keep Secrets Out of YAML

This one seems obvious and yet it gets violated constantly. I've reviewed pipelines where database passwords were in YAML comments, API keys were in environment variable values in the workflow file, and Docker registry credentials were committed directly.

Never put secrets in your pipeline YAML. Use your platform's secret store.

Wrong:

# Don't do this
env:
  DATABASE_URL: postgres://admin:mysecretpassword@db.example.com/myapp
  AWS_SECRET_KEY: AKIAIOSFODNN7EXAMPLE

GitHub Actions — correct approach:

name: Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: us-east-1

    - name: Build and push Docker image
      env:
        DATABASE_URL: ${{ secrets.DATABASE_URL }}
      run: |
        docker build \
          --build-arg DB_URL=$DATABASE_URL \
          -t myapp:${{ github.sha }} .
        docker push myapp:${{ github.sha }}

Secrets are set in GitHub → Settings → Secrets and variables → Actions. They're encrypted at rest, masked in logs, and never exposed in the YAML file itself.

GitLab CI equivalent:

deploy:
  stage: deploy
  script:
    - echo "$CI_REGISTRY_PASSWORD" | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  environment:
    name: production
    url: https://myapp.example.com

GitLab CI has built-in variables for the container registry (CI_REGISTRY_*) and commit information (CI_COMMIT_SHA). Use them.

Practice 2: Cache Dependencies Aggressively

The single biggest quality-of-life improvement in most pipelines is proper caching. Installing node_modules from scratch on every run is slow and unnecessary.

A Node.js project with 200 dependencies might take 45 seconds to install. With caching, that drops to 3 seconds. Multiply by 50 pipeline runs a day and that's 35 minutes of wasted time — per developer.

GitHub Actions — Node.js caching:

name: CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4

    - name: Setup Node.js
      uses: actions/setup-node@v4
      with:
        node-version: '20'
        cache: 'npm'  # This handles caching automatically

    - name: Install dependencies
      run: npm ci

    - name: Run tests
      run: npm test

The cache: 'npm' option in actions/setup-node@v4 handles cache key generation automatically — it uses the package-lock.json hash as the key, so the cache invalidates when dependencies change.

For Docker builds, use layer caching:

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3

    - name: Build Docker image
      uses: docker/build-push-action@v5
      with:
        context: .
        push: true
        tags: myapp:${{ github.sha }}
        cache-from: type=gha          # GitHub Actions cache
        cache-to: type=gha,mode=max

GitLab CI equivalent:

variables:
  npm_config_cache: "$CI_PROJECT_DIR/.npm"

cache:
  key:
    files:
      - package-lock.json
  paths:
    - .npm/

test:
  stage: test
  image: node:20-alpine
  script:
    - npm ci --cache .npm --prefer-offline
    - npm test

Practice 3: Parallelize Independent Jobs

Sequential pipelines are slow by default. If you're running lint, unit tests, integration tests, and security scans one after another — and they don't depend on each other — you're wasting time.

Sequential (slow, common):

lint (2min) → tests (8min) → security (3min) → build (4min) = 17 minutes

Parallel (fast, better):

lint (2min) ─┐
tests (8min) ├─→ build (4min) = 12 minutes
security (3min) ─┘

GitHub Actions makes this straightforward with the needs key:

name: CI Pipeline

on:
  pull_request:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: '20'
        cache: 'npm'
    - run: npm ci
    - run: npm run lint

  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: '20'
        cache: 'npm'
    - run: npm ci
    - run: npm test -- --coverage

  security:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - name: Run Trivy security scan
      uses: aquasecurity/trivy-action@master
      with:
        scan-type: 'fs'
        severity: 'HIGH,CRITICAL'

  build:
    runs-on: ubuntu-latest
    needs: [lint, test, security]  # Only runs if all three pass
    steps:
    - uses: actions/checkout@v4
    - name: Build Docker image
      run: docker build -t myapp:${{ github.sha }} .

  deploy-staging:
    runs-on: ubuntu-latest
    needs: [build]
    environment: staging
    steps:
    - name: Deploy to staging
      run: echo "Deploying to staging..."
      # Your actual deploy command here

needs: [lint, test, security] means the build job only starts when all three parallel jobs succeed. If any fails, the build is skipped.

Practice 4: Use Environments and Manual Approvals for Production

Deploying to production automatically on every merge to main is fine for some teams. For others — especially those with compliance requirements or complex rollback scenarios — requiring human approval makes sense.

Both GitHub Actions and GitLab CI handle this natively.

GitHub Actions with environment protection rules:

  deploy-production:
    runs-on: ubuntu-latest
    needs: [deploy-staging]
    environment:
      name: production
      url: https://myapp.example.com
    steps:
    - name: Deploy to production
      run: |
        kubectl set image deployment/myapp myapp=myapp:${{ github.sha }}
        kubectl rollout status deployment/myapp

In GitHub, configure the production environment with required reviewers. When the job reaches this stage, it pauses and sends notifications to reviewers. Nobody can approve their own deployment. Approved → job runs. This is a repo settings change, not a YAML change.

GitLab CI with manual jobs:

deploy-production:
  stage: deploy
  script:
    - kubectl set image deployment/myapp myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  environment:
    name: production
  when: manual     # Requires someone to click "Play" in the GitLab UI
  only:
    - main

Practice 5: Fail Fast, Fail Clearly

Pipelines that silently fail, or fail in ways that make debugging a half-hour ordeal, destroy developer trust. Once developers stop caring if the pipeline is green, the whole system breaks down.

Three sub-practices here:

Run the cheapest checks first. Linting takes 10 seconds. Unit tests take 2 minutes. Integration tests take 10 minutes. Put linting first — if the code has syntax errors, there's no point waiting 12 minutes to find out.

Use clear, meaningful job names. build tells you nothing when it fails. build-docker-image-production tells you exactly what broke.

Set timeouts on jobs. A hung test suite can tie up runners for hours without timeouts:

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 15    # Kill the job if it takes longer
    steps:
    - run: npm test

Post failure notifications where developers actually look:

  notify-failure:
    runs-on: ubuntu-latest
    needs: [test, build]
    if: failure()   # Only runs if previous jobs failed
    steps:
    - name: Send Slack notification
      uses: slackapi/slack-github-action@v1.26.0
      with:
        payload: |
          {
            "text": "Pipeline failed on ${{ github.ref }} — ${{ github.event.head_commit.message }}"
          }
      env:
        SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

Tool Comparison

GitHub Actions wins on ecosystem size and free minutes for private repos. GitLab CI wins if you want everything — code, CI, registry, security scanning — in one platform. CircleCI is excellent but harder to justify the cost when GitHub Actions exists.

Common Mistakes I See Constantly

Using main branch triggers for every job. If a PR pipeline runs lint and tests, the post-merge pipeline on main probably doesn't need to re-run those same checks. Run them on PR, deploy on merge to main.

Not pinning action versions. uses: actions/checkout@main means your pipeline can break overnight if that action changes. Always pin to a specific version or commit SHA: uses: actions/checkout@v4.

Ignoring security scanning. Adding a one-line Trivy or Snyk scan to your pipeline takes 10 minutes and catches known vulnerabilities in dependencies. There's almost no excuse not to.

Not testing the rollback. Most teams test the deploy. Very few teams test that rollback actually works until they need it in an incident. Test it in staging before you need it in production.

Connecting CI/CD to the Rest of Your Stack

A good pipeline is only valuable if what it deploys is solid. For containerised apps, Docker for backend developers covers multi-stage builds and production-grade Dockerfiles. Your pipeline builds those images.

If you're deploying to Kubernetes, the deploy Node.js to Kubernetes guide covers the kubectl commands your pipeline will run. And for infrastructure provisioning, Terraform vs Pulumi vs CloudFormation covers automating the environment your pipeline deploys into.

Your pipeline also benefits from version control hygiene — if you're newer to git workflows, the git and GitHub guide for beginners is a solid foundation.

Conclusion

Five practices: secrets in secret stores, cache aggressively, parallelize independent jobs, protect production with approvals, and fail fast with clear messages. None of these are complicated in isolation. The challenge is implementing all five from day one rather than retrofitting them after they've been painful for six months.

Start simple. A working pipeline that deploys reliably is better than a sophisticated pipeline that's too complex to maintain. Add sophistication as the team's needs grow. The goal is confidence — confidence that a green pipeline means the code actually works.

FAQ

How long should a CI/CD pipeline take? For most web applications, target under 10 minutes for the full pipeline from push to deploy. If your tests alone take 15+ minutes, developers start skipping them or waiting too long for feedback. The fast feedback loop is the entire point. Split long test suites into parallel jobs, cache aggressively, and consider whether every test needs to run on every push vs only on pull requests.

Should I use GitHub Actions or GitLab CI for a new project? If your code lives on GitHub, GitHub Actions is the obvious choice — no extra infrastructure, great marketplace, and tight integration. If you need self-hosted runners, prefer a single tool for all DevOps (pipelines, container registry, issue tracking), or need advanced compliance features, GitLab CI is worth the setup. Both are excellent. The choice should follow where your code lives, not the other way around.

What is the difference between a CI pipeline and a CD pipeline? CI (continuous integration) is the automated build and test phase that runs on every commit or pull request. It verifies that new code doesn't break existing functionality. CD (continuous delivery or deployment) takes a verified build and delivers it to an environment — staging for continuous delivery, production for continuous deployment. Most teams implement CI first, then add CD once their tests are reliable enough to trust.