.dockerignore and Layer Caching: Faster Docker Builds in Dev and CI

The hidden cost of slow image builds

A ten-minute image build looks like a minor irritation. Multiply it by every commit, every engineer and every pipeline retry and it becomes a tax on your whole delivery process. Developers context-switch while they wait. Hotfixes take longer than they should. CI minutes grow. And the build often rebuilds things that have not changed at all, because a cache is being invalidated for no good reason.

Caching is a mechanical topic, and it rewards care. It is one of the highest-return pieces of our Docker containerisation service, and it feeds straight into your CI/CD pipeline.

How Docker caching actually works

Each instruction in a Dockerfile creates a layer. For each instruction, the builder computes a cache key from the instruction itself and, for COPY and ADD, the content of the files involved. If the key matches a cached layer, it is reused. If any layer changes, every layer after it is rebuilt.

Two consequences follow. First, order matters: put things that rarely change at the top. Second, the files you send to the builder matter: anything in the build context can alter a COPY key.

Step one: the .dockerignore

The build context is everything sent to the builder. Without a .dockerignore, that includes .git, node_modules from the host, build output and any local environment file. A typical file looks like this:

.git
.gitignore
node_modules
dist
build
coverage
*.log
.env
.env.*
.vscode
.idea
Dockerfile
docker-compose*.yml
README.md

This reduces upload time, stops host-side changes from busting caches, and prevents secrets in a local .env from being copied into an image. Be careful with patterns: a rule that excludes too much causes confusing COPY failures, so we test the build after changing it. For monorepos we ignore everything and explicitly allow only what each service needs.

Step two: instruction order

The classic fix is copying dependency manifests first:

# syntax=docker/dockerfile:1
FROM python:3.12-slim AS build
WORKDIR /app

# 1. Rarely changes: dependency manifests
COPY requirements.txt ./
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

# 2. Changes constantly: application source
COPY src ./src

With this order, changing application code reuses the dependency layer. Changing requirements.txt rebuilds only from that point. The same idea applies to package.json with npm ci, go.mod with go mod download, or pom.xml with a dependency resolve step. The wider structure of stages is covered in Dockerfile multi-stage builds.

Step three: cache mounts

A cache mount keeps a directory such as the pip or npm cache on the builder between runs, without writing it into the image. Even when a lockfile changes and the install layer must rerun, packages are not downloaded again. This requires BuildKit, which is the default builder in current Docker releases.

Step four: caching in CI

Here is the part many teams miss. A CI runner is typically a fresh machine, so a local cache does nothing. You need a cache that survives between jobs.

- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v6
  with:
    context: .
    push: false
    tags: myapp:test
    cache-from: type=gha
    cache-to: type=gha,mode=max

For GitLab CI or self-hosted runners, a registry cache (type=registry,ref=...) stored in your private repository works well, as described in private registry setup. Persistent runners with a local BuildKit daemon are another option and are fastest when you control the infrastructure.

Reading the build output

You cannot tune what you cannot see. BuildKit’s plain progress output shows each step with a CACHED marker or its duration:

docker build --progress=plain -t myapp:test . 2>&1 | tee build.log

Change one source file, rebuild, and read the log from the top. The first step that is not CACHED is where invalidation starts. If it is the dependency install, something earlier than it changed: usually a COPY that includes more than the manifest, or a file in the context that changes on each run. This simple loop finds the cause of most slow builds in minutes.

Language specifics worth knowing

The pattern is the same everywhere, but the details differ:

• Node.js: copy package.json and the lockfile, run npm ci, then copy source. Mount the npm cache.
• Go: copy go.mod and go.sum, run go mod download, then copy source. Mount the module and build caches.
• Java with Maven or Gradle: resolve dependencies in a step that depends only on the build files, and mount the local repository cache.
• Monorepos: copy only the manifests for the one service being built, so that a change in another service does not invalidate this one.

Caching and security

A cache is a place where old content lives. A secret that was ever copied into a cached layer can persist in the cache backend long after the Dockerfile changed. That is another reason to use secret mounts rather than ARG or COPY for credentials, and to restrict who can read a registry cache. We also keep the cache repository separate from release repositories, so that a cache image is never mistaken for something deployable.

Decisions and trade-offs

GitHub Actions cache vs registry cache. The Actions cache is easy but has size limits and eviction. A registry cache has no such limit, works across CI systems and developer machines, and costs storage. We choose based on image count and team size.

mode=max or mode=min. Max caches intermediate stages, so multi-stage builds benefit more, at the cost of a larger cache. For stages that do heavy compilation, it is worth it.

Cache vs reproducibility. Aggressive caching can hide breakage that a clean build would reveal. We run a scheduled uncached build to catch it.

When not to bother. If your builds take thirty seconds and run a few times a day, leave them alone.

Common failure modes

• Cache never hits because the key includes a changing value like a build timestamp or git SHA written into an early layer.
• COPY . . before dependencies, which invalidates the install layer constantly.
• Secrets in layers. Never pass credentials through ARG or copy them in. Use RUN --mount=type=secret.
• Stale apt-get update layers cached for months. Combine update and install in one RUN, and rebuild periodically.
• Cache growth that slows import. Expire it.

Delivery and handover

You receive a reviewed .dockerignore, restructured Dockerfiles, CI cache configuration and a one-page explanation of how to read build logs for cache hits. We pair this with image size reduction because the same changes help both.

How to tell it is working

Measure wall-clock time for three cases: a no-change rebuild, a one-line code change and a dependency update. A healthy setup shows seconds for the first, a short build for the second, and a moderate rebuild for the third. CI logs show CACHED against the early layers. To have your builds profiled, contact us.

References

• Docker build cache, Docker Docs
• Build context, Docker Docs
• Cache management with GitHub Actions, Docker Docs