The tension every security leader knows
SOC teams are pushed to automate everything to address analyst capacity constraints. Auditors and risk committees demand defensible evidence trails. These objectives are not inherently in conflict — but vendor automation platforms often optimise for the former at the expense of the latter.
The failure pattern is consistent: a SOC invests in SOAR automation, closes large numbers of alerts without analyst review, and then faces a post-incident review or regulatory inspection asking for evidence of each action taken. The evidence doesn't exist in usable form. Screenshots in tickets, Slack threads, and undocumented playbook runs are not evidence — they are noise.
This is written for CISOs and detection engineering leads who need automation that survives a post-incident review, a regulatory inspection, and a procurement replay six months after implementation.
The evidence primitive as a design requirement
Every automated action in a SOC environment should emit a structured evidence record containing:
- Who or what initiated the action — analyst name and role, or playbook name and version
- What action was taken — specific command, API call, or configuration change with parameters
- What the pre-action state was — network state, process list, account status before the action
- When the action occurred — timestamp with timezone, correlated to the SIEM event timeline
- Policy authority — which approved policy or playbook version authorised this action
- Observable outcome — what changed as a result, with artefact hashes where applicable
If your SOAR platform cannot emit these fields as structured data, you are not ready for closed-loop automation. You can build assisted triage workflows — the analyst sees the draft action and approves it — but autonomous execution requires evidence capability first.
Defining the human gate policy
Not all SOC actions carry the same risk profile. The distinction between human-approved and machine-executable actions should be encoded in written policy, reviewed by risk and legal, and reflected in playbook configuration — not left to analyst discretion under pressure.
Actions that should always require human approval (examples, not exhaustive):
- Network isolation or quarantine of production hosts
- Account disable or privilege revocation
- Customer-visible communications (takedown requests, breach notifications)
- Actions that touch regulated data stores or backup infrastructure
- Firewall rule changes that could affect availability
Actions appropriate for machine execution with audit trail:
- Alert enrichment (WHOIS lookups, threat intelligence queries, asset classification)
- Ticket creation and initial triage documentation
- Isolation of non-production or sandbox systems
- Threat intelligence correlation and case linking
Write the policy in language that your SOAR platform can enforce. "A senior analyst must approve network isolation" is not enforceable if the playbook can bypass it. The governance has to be in the workflow, not in the minds of the people running it.
Dry-run mode and the automation readiness test
Before enabling any new automated playbook in production:
- Run in dry-run mode — log every action the playbook would take without executing it. Review the logs for unexpected scope (what systems would be touched that shouldn't be) and coverage gaps (cases where the playbook wouldn't fire but should).
- Stress analysts, not production — simulate the playbook under realistic alert load using historical or synthetic data. Analysts who will be working with the automation in production should review dry-run outputs before go-live.
- Define the off-switch — document who can disable the playbook under attack, and how. "Automation off" drills should be run quarterly. The worst failure mode is automation you cannot safely halt when it is behaving unexpectedly during an incident.
Retention architecture for regulated enterprises
Evidence retention requirements vary by regulatory framework. UAE Central Bank cybersecurity requirements, EU NIS2, and UK FCA regulations each specify minimum retention periods and access control requirements for incident evidence.
Design retention as an architecture requirement from the start:
- Immutable storage for closed-loop action logs — write-once, read-many, with deletion restricted to documented retention expiry
- Access controls aligned with legal hold procedures — evidence that may be relevant to litigation must be preserved under different access rules than standard operational logs
- Structured format that is searchable and reproducible — auditors need to query the evidence; flat log files they cannot parse are not useful evidence
Consult legal counsel on specific retention periods before implementation. Implementing a retention architecture that does not meet regulatory minimums creates both compliance exposure and operational risk.
The "automation off" discipline
High-maturity SOC programmes rehearse operating without automation at least quarterly. The goal is not to prove automation is unreliable — it is to ensure analyst capability does not atrophy to the point where a SOAR outage becomes a security incident.
Automation dependency creates single points of failure. An organisation whose analysts cannot triage alerts manually within 30 minutes of SOAR unavailability has an automation-induced risk that didn't exist before the automation was implemented.
Key takeaways
- Treat evidence fields as first-class schema — design them into the automation platform before production use.
- Separate assisted triage from autonomous remediation in written policy, enforced by playbook configuration.
- Run dry-run mode before any new automated playbook goes live; stress analysts in controlled conditions.
- Rehearse "automation off" operations quarterly — the capability to operate without SOAR is a security control.
- Align retention periods and access controls with legal and compliance teams before scaling alert volume.
For common buyer questions, see the FAQ on this page.
Sources: NIST SP 800-61r3 incident response, MITRE ATT&CK, verified 10 August 2026.
Related: Threat-informed detection engineering in practice · csoc SIEM/XDR — how we built evidence-first architecture
