Where image pipelines usually go wrong
Image builds tend to grow by accretion. Someone writes a workflow that builds and pushes latest. Another team copies it and adds a scan step. A third repository stores an AWS access key as a CI variable because the original author left. After a year you have a dozen near-identical pipelines, long-lived credentials in several places, tags that mean different things in different repositories, and no one sure which commit produced the image in production.
Pipeline integration fixes this by treating the image build as a standard, owned product. It is part of our Docker containerisation service and connects to our wider CI/CD pipeline work.
What good looks like
- Every merge to the main branch produces exactly one image with a traceable tag.
- Pull requests build (and scan) but do not push to production repositories.
- The pipeline authenticates with OIDC, not stored keys.
- Caching makes typical builds fast.
- Build, scan, push, sign and deploy trigger are one coherent flow.
- Pipelines are defined once and reused.
A reusable GitHub Actions workflow
This example builds and pushes to Amazon ECR using OIDC. The structure is the same for other registries, only the login step changes.
name: container-build
on:
workflow_call:
inputs:
image-name: { required: true, type: string }
secrets: {}
permissions:
id-token: write
contents: read
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/ci-image-push
aws-region: eu-west-1
- id: ecr
uses: aws-actions/amazon-ecr-login@v2
- uses: docker/setup-buildx-action@v3
- id: build
uses: docker/build-push-action@v6
with:
context: .
push: ${{ github.ref == 'refs/heads/main' }}
tags: ${{ steps.ecr.outputs.registry }}/${{ inputs.image-name }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
The account id above is a documentation placeholder. The role it assumes is allowed to push to named repositories only, and its trust policy restricts which repositories and branches may assume it. Callers use uses: ./.github/workflows/container-build.yml (or a central repository) and pass the image name. Scanning is added as described in Trivy in CI/CD, and signing as in image signing.
The GitLab CI equivalent
GitLab offers a built-in container registry and CI variables for authentication. A typical job uses docker buildx build or a rootless BuildKit or Buildah setup on a runner. Reusable pipeline logic goes into a shared include: file, versioned by tag. If your runners use Docker-in-Docker, we review the isolation and consider a safer builder.
Tagging and promotion
A scheme that works well:
| Tag | Meaning | Mutable? |
|---|---|---|
<git-sha> | Exact commit, always pushed | No |
v1.4.2 | Release, created from a git tag | No |
main | Latest build of main, for convenience | Yes, never used in production |
Promotion between environments should move the same image forward, not rebuild it. Staging and production run identical bytes, which is the point. Deployment can then be driven by GitOps with Argo CD or Flux, where the pipeline’s last job opens a pull request or commits the new tag to the manifest repository.
The trust policy that makes OIDC safe
OIDC removes stored keys, but only if the cloud side is strict. The role’s trust policy should name the repository and, ideally, the branch or environment allowed to assume it. A policy that accepts any repository in your GitHub organisation means a compromised or careless repository can push images to production repositories. A narrower condition looks like this:
{
"Effect": "Allow",
"Principal": { "Federated": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com" },
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
"token.actions.githubusercontent.com:sub": "repo:acme/api:ref:refs/heads/main"
}
}
}
Again the account id is a documentation placeholder. We keep one push role per service or per team, so the blast radius of a single compromised workflow stays small, and we manage the roles in Terraform.
Pull requests from forks and untrusted code
Pull request builds are where pipelines get attacked, because they run code that a contributor controls. Our rules are simple: forked pull requests get no secrets and no cloud role, builds on pull requests never push to the production repository, and any workflow that uses pull_request_target is reviewed by hand. If you accept outside contributions, this is worth a careful look even if the rest of your pipeline is sound.
Failing the right way
A pipeline that is red for the wrong reasons trains people to ignore it. We separate hard failures (build errors, critical vulnerabilities with a fix available, signing errors) from soft ones (registry slowness, which gets a bounded retry). Flaky steps are logged and fixed rather than rerun until they pass.
Speed
Most of the speed comes from caching and avoiding unnecessary builds:
- Use layer caching and a tight .dockerignore.
- Use path filters so a README change does not rebuild every image in a monorepo.
- Run scans and tests in parallel where they are independent.
- Use larger or self-hosted runners only when profiling shows CPU is the bottleneck.
Decisions and trade-offs
Hosted or self-hosted runners. Hosted runners are simple but start cold, with a cache restore each run. Self-hosted runners keep BuildKit state warm and are faster, but you own patching and isolation. We usually start hosted and move only when build times justify it.
Multi-platform images. Emulation makes ARM builds slow. Native runners or cross-compilation avoid that. Only build platforms you actually deploy.
Build once, deploy many. Environment differences belong in configuration, not in separate images.
Jenkins. If you are still on Jenkins, the same principles apply, and we can help with moving from Jenkins to GitHub Actions if that is the direction.
Common failure modes
- Long-lived registry credentials in CI variables, shared across projects.
- Pushing from pull requests from forks, which can leak secrets or pollute the registry.
latestin production manifests.- Cache poisoning or stale cache hiding breakage. We run periodic clean builds.
- Unpinned actions from third parties. We pin by version or commit SHA and let Renovate update them.
Delivery and handover
We deliver the reusable workflow or include, registry and OIDC configuration (as Terraform where possible), tagging rules, and a short guide for adding a new service. Your team can then onboard repositories by adding a few lines.
How to tell it is working
Image builds are fast and predictable, any running container maps to a commit, no long-lived registry credentials exist in CI, and adding a new service to the pipeline takes minutes. To plan your pipeline, book a consultation.