Why compliance by spreadsheet fails
The usual pattern: a customer questionnaire or auditor asks whether your servers follow the CIS benchmark. An engineer spends a fortnight running a scanner, fixing what they can by hand, and pasting results into a document. Three months later new servers have joined, someone has relaxed a setting to debug a problem and left it, and the document is fiction.
Automation changes the economics. If controls are expressed as Ansible tasks, you can check them on a schedule, fix drift, and generate evidence for any date. This page explains how we deliver that within our Ansible configuration management service.
What good looks like
- A current score and a list of failing controls per host, produced without manual effort.
- Remediation is code: reviewed, tested, repeatable.
- Every exception is recorded with a reason, a risk owner and a review date.
- New servers are compliant from first boot, using the baseline role as the starting point.
- Auditors receive evidence in a consistent format, not a screenshot.
How it works in practice
Audit first, never remediate blind
Each CIS recommendation has an audit step and a remediation step. We implement them separately, so the same role can run in two modes. A variable toggles between them:
- name: 5.2.4 Ensure SSH access is limited (audit)
ansible.builtin.command: sshd -T
register: sshd_effective
changed_when: false
check_mode: false
tags: [cis, audit, ssh]
- name: 5.2.4 Record result
ansible.builtin.set_fact:
cis_results: "{{ cis_results | default([]) + [{
'control': '5.2.4',
'host': inventory_hostname,
'pass': ('allowusers' in sshd_effective.stdout | lower) or
('allowgroups' in sshd_effective.stdout | lower)
}] }}"
tags: [cis, audit, ssh]
Control numbers differ between benchmark versions and distributions, so we take them from the specific benchmark document you are measured against rather than assuming. The audit pass makes no changes, so it is safe to run on production at any time.
Choosing what to remediate
A first audit commonly returns a long list. We sort it:
- Safe and mechanical. File permissions, banner text, disabling unused filesystems. Apply in bulk after testing.
- Needs testing. Kernel parameters, mount options such as
noexecon/tmp, firewall defaults. Test against real workloads. - Needs an exception. Controls that conflict with how your application works. Documented, not ignored.
- Structural. Separate partitions for
/var/logon running hosts. Fixed at the image or rebuild stage, not in place.
Tools beyond Ansible tasks
Two standard tools complement Ansible. OpenSCAP, with the SCAP Security Guide content, scans a host against a profile and produces an HTML or XML report, which is useful as independent evidence. Lynis is an auditing tool that gives a broader hardening view. CIS also provides its own assessor tool, CIS-CAT, whose use depends on membership and licensing terms, so we check what you are entitled to. A common design is Ansible for remediation and for scheduled audit tasks, with OpenSCAP as the second opinion.
Running a scan from a playbook looks like this:
oscap xccdf eval \
--profile xccdf_org.ssgproject.content_profile_cis_server_l1 \
--results results.xml --report report.html \
/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml
Profile and datastream names depend on the distribution and package version, so we confirm them on a test host first. The playbook then fetches the result files to the controller and archives them.
Report generation
Results from each host are collected and rendered with a Jinja2 template into a summary: overall pass rate by host group, failing controls by frequency, and the exception register. Raw data is also stored as JSON so it can be graphed over time in Grafana and queried when someone asks what the state was on a given date. Trend lines matter more than any single score.
Exceptions as data
Exceptions live in variables, not in people’s heads:
cis_exceptions:
- control: "1.1.2.1"
hosts: [db01]
reason: "Dedicated partition not possible without rebuild; scheduled for Q1 image refresh"
owner: "platform-lead"
review: "2027-01-31"
The audit report lists them, so reviewers see accepted risk next to open gaps.
Decisions and trade-offs
Level 1 or Level 2. Level 1 is almost always the right starting point. Level 2 buys extra assurance at the cost of operational friction. Apply it where the data justifies it.
Remediate in place or rebuild from images. For long-lived servers, in-place remediation is the only option. For disposable ones, hardening the image is cleaner, and Ansible can run inside the image build.
Own roles or community roles. Community roles save time. They also change with upstream benchmark updates, so we pin versions and review changes before adopting them.
Compliance versus security. Meeting a benchmark does not make a system secure. It is a floor. We say so plainly, and when needed we recommend broader controls, such as the cloud checks in our managed DevOps compliance work.
Common failure modes
- Hardening that breaks the application. Mitigated by staged rollout and tests.
- One-off compliance. The score decays. Scheduled audits and drift alerts prevent it.
- Over-trusting a score. A green report with exceptions hidden. The exception register is part of the report.
- Benchmark version drift. The OS is upgraded and the controls no longer map. We track versions per distribution.
- Reports no one reads. We agree the audience and cadence before building the format.
Delivery and handover
You receive the audit and remediation roles, scheduled scans (in AWX if you run it), the report templates, the exception register format, and documentation that explains how to add a control or an exception. The documentation work keeps control mappings readable to non-specialists. We do not certify anyone: we provide the technical evidence and automation.
A note on exceptions
Some controls will conflict with how a workload really runs. Record each exception as a variable override with a reason, an owner and a review date, so auditors see a decision rather than a gap.
How to tell it is working
- The pass rate rises, then stays stable, rather than spiking before each audit.
- Drift is detected within one scan interval and corrected.
- A new server’s first scan has few failures.
- The exception list is short, current and owned.
To find out where your servers stand today, request a compliance gap review.