What an auditor (or an incident) will ask
Sooner or later someone asks: who deleted that database snapshot, when did this role gain admin rights, and who read that customer export last quarter? If the answer needs a week of archaeology, or the evidence was overwritten, the logging is not doing its job. Audit logs are the record of administrative and access actions, and unlike debug logs, their value comes from being complete, trustworthy and available when needed.
This work belongs in our wider logging, ELK, Loki and OpenTelemetry service, and it supports compliance programmes such as CIS-based hardening under Managed DevOps. We do not certify anything ourselves. We build the technical evidence trail.
What good looks like
- Control plane activity from every account and subscription is captured, including read access to sensitive resources where required.
- Logs are delivered to a separate, restricted location that the monitored accounts cannot modify.
- Records are retained for the agreed period and are tamper-evident.
- Authorised people can answer “who did what, when” in minutes.
- Risky actions, such as disabling logging or creating an access key for the root user, raise alerts.
What to collect
| Source | What it records | Typical destination |
|---|---|---|
| AWS CloudTrail | API calls across AWS accounts, management and optionally data events | S3 in a log archive account |
| Azure Activity Log and diagnostic settings | Subscription level operations and resource logs | Log Analytics workspace and storage account |
| Kubernetes audit log | Requests to the API server, with who, what and response | Collector to Loki or Elasticsearch |
| Identity provider | Sign-ins, MFA events, role and group changes | SIEM or log platform |
| Databases and storage | Access to sensitive tables and objects | Log platform and archive |
| Your application | Business audit events such as permission changes | Dedicated audit index or stream |
How it works in practice
AWS: organisation trail to a log archive account
With AWS Organizations, an organisation trail records events for all member accounts into one S3 bucket in a dedicated log archive account. Key settings are multi-region coverage, log file validation, KMS encryption and a bucket policy that allows CloudTrail to write and almost nobody to delete.
resource "aws_cloudtrail" "org" {
name = "org-audit"
s3_bucket_name = aws_s3_bucket.audit_logs.id
is_organization_trail = true
is_multi_region_trail = true
include_global_service_events = true
enable_log_file_validation = true
kms_key_id = aws_kms_key.audit.arn
}
The bucket should have versioning, encryption and, where your requirements demand write-once storage, S3 Object Lock configured at bucket creation. Lifecycle rules move older logs to colder classes. The structure is created with Terraform so that it is reviewed and reproducible, and a multi-account setup is easier to do before workloads sprawl.
Azure: diagnostic settings
In Azure, the Activity Log is exported with diagnostic settings to a Log Analytics workspace for query and to a storage account for retention, possibly with immutability policies. Resource logs (Key Vault, SQL, storage) need their own diagnostic settings, which is easy to miss, so we enforce them with Azure Policy and code. This pairs with our Azure DevOps consulting work.
Kubernetes audit logs
The API server writes audit events according to an audit policy. A policy that logs everything at request body level is huge, so we tune levels: metadata for most resources, request bodies for sensitive ones, and omit noisy reads such as health checks. On managed Kubernetes (EKS, AKS, GKE) audit logs are delivered through the provider’s logging feature and then routed to your platform. See the RBAC glossary entry for what these events relate to.
Making it searchable
Files in a bucket are an archive, not an investigation tool. We provide a query path: Athena over CloudTrail in S3, Log Analytics queries in Azure, or ingestion of selected events into Elasticsearch or Loki through the patterns on the log routing page. Fields such as actor, action, resource, source_ip and outcome are normalised so one query works across clouds.
Integrity and alerts
Log file validation in CloudTrail lets you prove files were not altered after delivery. Access to the audit store is logged and alerted. We also alert on actions that weaken the trail itself: stopping a trail, changing a bucket policy, disabling diagnostic settings, or deleting a key. Alert design follows the principles of log-based alerting.
Decisions and trade-offs
Data events or management events only. Management events are cheap and essential. Data events (object-level S3 access, for example) can generate huge volume and cost, so enable them for specific sensitive buckets, not everything.
Search platform or query-in-place. Query-in-place with Athena or Log Analytics is cheaper for rare investigations. A log platform is better for frequent search and alerting.
Immutability and the right to erasure. Write-once storage can conflict with deletion duties for personal data. Audit logs should minimise personal data and avoid content, and the question should be agreed with your legal advisers, as discussed on the log scrubbing page.
When this is too much. An early stage company with no compliance commitments may only need CloudTrail enabled with sensible retention. We scale to what is needed.
Common failure modes
- Logging enabled in some regions or accounts only, leaving blind spots.
- Logs stored in the same account the attacker or an administrator controls.
- No restore or query test, so the first search happens under audit pressure.
- Retention shorter than the commitment because a lifecycle rule was copied from an application bucket.
- Cost surprise from data events or verbose Kubernetes audit levels.
Delivery and handover
You receive the Terraform code, a source inventory with owners, retention settings with their rationale, sample queries for typical audit requests, and alerts for tampering. We walk your security lead through the evidence so they can explain it to an auditor.
How to tell it is working
Pick a random administrative action from last month and trace it to a person within minutes. Confirm every account appears in the trail, validation passes, and an attempt to change the audit bucket as an ordinary admin is blocked and alerted. To scope this for your frameworks, book a consultation.