What Azure Pipelines is and who it suits
Azure Pipelines is the CI/CD component of Azure DevOps, sitting beside Boards, Repos, Test Plans and Artifacts. A pipeline is defined in a YAML file, triggered by commits or schedules, and executed by agents. It builds almost any language, can target any cloud or on-premises system, and has particularly smooth integration with Azure resources.
It fits best when your organisation already plans work in Azure DevOps, wants approvals and audit trails that governance teams recognise, or deploys mostly to Azure. Our Azure Pipelines consulting work usually involves cleaning up a mix of classic release pipelines, copy-pasted YAML and a handful of service connections with far too much access.
How we structure pipelines in production
Templates and the extends model
The single most useful design decision is a central template repository. Application pipelines do not define steps from scratch; they extend a template that enforces build, scan and publish, and accept parameters for what varies.
# azure-pipelines.yml in an application repository
trigger:
branches:
include: [main]
resources:
repositories:
- repository: templates
type: git
name: Platform/pipeline-templates
ref: refs/tags/v4
extends:
template: standard-service.yml@templates
parameters:
serviceName: orders-api
deployEnvironments:
- name: staging
- name: production
Because the pipeline extends the template, required steps such as image scanning cannot be quietly removed by an application team. Tagging the template repository lets the platform team roll out changes deliberately. This mirrors the approach in our reusable workflows and pipeline as code work.
Environments, approvals and checks
Deployment jobs target environments. An environment can require approvals, restrict branches, apply business hours checks, and hold exclusive locks so two deployments do not collide. It also records what was deployed and when. We map environments to your actual release process and keep production approvers as a named group, not individuals. See environment promotion gates and approvals.
Build once, promote the artefact
The pipeline builds an image or package once, then promotes the same artefact through environments. Rebuilding for each stage is a common source of “it worked in staging” surprises. We tag images from the commit and deploy by digest.
Deployment strategies and verification
Deployment jobs support run-once, rolling and canary strategies, with lifecycle hooks for pre-deploy, routing and post-deploy steps. We pair them with smoke tests that run immediately after release and with automated rollback to the previous artefact when checks fail. For regulated teams, the same environment history gives an audit trail of what was deployed, by which pipeline run, approved by whom. Keeping verification inside the pipeline means a bad release is caught in minutes, not by the first customer to notice.
Security and access
- Workload identity federation. Service connections to Azure use federated credentials, so no client secret is stored or rotated. Scope each connection narrowly.
- Key Vault for secrets. Pipelines read secrets at run time from Azure Key Vault instead of long-lived variable group values.
- Pipeline permissions. Restrict which pipelines may use each service connection, agent pool and variable group, and avoid “grant access to all pipelines”.
- Protected branches and build validation. Pull request policies require a successful build and review before merging.
- Fork and external contribution settings. Control whether secrets are exposed to builds from forks.
- Least-privilege identities. Project build service accounts get only the repository and artefact access they need.
Teams deploying to clusters should also read our guidance on Azure DevOps consulting for the wider organisation-level setup.
Cost and capacity
Cost control starts with knowing which pipelines consume the most minutes.
Parallel jobs are the capacity you pay for with Microsoft-hosted agents, and extra parallelism is a licensing decision. Before buying more, reduce waste: skip jobs for unchanged paths, cache package restores, cancel superseded runs, and split long suites sensibly. Self-hosted agents add infrastructure cost but may reduce per-minute spend and allow private access. Use scale-set agents so capacity expands under load and returns to zero after, and keep them ephemeral so one job cannot influence the next.
Day-2 operations
- Template versioning: treat the template repository as a product, with changelogs and a deprecation policy.
- Agent upkeep: update agent software and images regularly, and monitor queue time per pool.
- Housekeeping: set retention policies on runs and artefacts, review stale pipelines and prune unused service connections.
- Auditing: use audit logs to see who changed permissions or approvals.
Failure modes we design around
- Over-privileged service connections that can touch every subscription. We scope to a resource group and use federation.
- Classic release pipelines nobody dares edit. We convert them to YAML in stages.
- Approvals tied to one person who goes on leave. We use groups with a backup.
- Shared self-hosted agents with leftover state. We move to ephemeral scale-set agents.
- Variable group sprawl with duplicated secrets. We centralise in Key Vault.
When to choose something else
If your code lives in GitHub, GitHub Actions is often the simpler route, and Azure Pipelines can still be used with GitHub repositories if you value its governance features. If you run GitLab, use its native CI. For complex estates still on Jenkins, plan a gradual move. For Azure AKS delivery, we often keep Azure Pipelines for build and test and let ArgoCD or Flux apply changes, which keeps cluster credentials out of the pipeline.
For a full walkthrough, read our Azure DevOps pipelines guide, see our CI/CD pipeline service, or book a consultation.
Official documentation: learn.microsoft.com/en-us/azure/devops/pipelines