The Analyst Role Shifts
From Queue-Clearing to Verification
When the machine drafts the disposition, the analyst's job inverts. Tier-one work has historically been clearing a queue: open the alert, gather context, decide, close, repeat. Assisted triage arrives with the context gathered and the decision drafted, and the human contribution becomes verification — is this recommendation actually right? That is a harder skill than it sounds, because a fluent, confident, well-formatted summary reads as correct whether or not it is. Good verification means checking the claimed evidence against the raw telemetry, noticing what the summary does not mention, and knowing when an alert deserves investigation beyond what the AI did. Analysts who only ever confirm are not verifying; they are decorating an automated pipeline with a headcount.
- The scarce skill shifts from processing speed to catching wrong recommendations
- Fluent and confident output reads as correct even when it is not
- Real verification samples raw evidence, not just the AI's narrative of it
- An analyst who never disagrees with the tool is a rubber stamp, not a control
Trust Calibration Cuts Both Ways
Automation bias is the famous failure: the tool is right often enough that humans stop checking, and approval becomes reflex. But the opposite failure is just as real and gets less attention — an analyst burned once by a bad recommendation may quietly re-verify everything, at which point the SOC pays for the AI and the full manual workload plus the overhead of reconciling them. Both failures are invisible in aggregate dashboards and visible in individual behavior: watch per-analyst override rates. Near zero suggests rubber-stamping; very high suggests distrust or a genuinely poor tool — the rate alone cannot say which, so read it alongside the sampled-dismissal reviews from the metrics work. Calibrated trust is the real product of the adoption period, and it must be managed, not assumed.
- Automation bias: high accuracy trains humans to stop verifying — accuracy causes the complacency
- Distrust bias: full re-verification of every output erases the efficiency gain entirely
- Per-analyst override rates surface both failure modes; team averages hide them
- Read override rates against sampled ground truth before concluding anything
The Ladder Problem
Tier-one triage has been the industry's apprenticeship: repetitive, yes, but the repetition is how junior analysts built intuition for what normal looks like in real telemetry. Automate that layer and you have removed the bottom rungs of the ladder while still needing people at the top — senior analysts capable of judging the AI's output, a skill built from exactly the reps the AI now does. No one has fully solved this. Partial answers exist: rotate juniors through the dismissed-alert sampling so they still work raw cases against ground truth; have them investigate first and compare against the AI's take rather than reading it first; treat some manual triage as deliberate training volume, inefficiency accepted on purpose. What fails is pretending the ladder is intact.
- Tier-one repetition was the apprenticeship, not just the toil
- Verification skill at the top depends on reps the AI now absorbs at the bottom
- Dismissal-sampling duty doubles as ground-truth training for juniors
- "Investigate first, then compare" preserves learning that reading-first destroys
Training Has to Change Shape
A training program built for the pre-AI SOC teaches alert-processing procedure: which console, which query, which disposition codes. The assisted SOC needs three additions. First, evidence-checking as an explicit, practiced skill — exercises where analysts receive plausible AI dispositions, some deliberately wrong, and must find the flaws; if your training set contains no wrong recommendations, you are training compliance, not verification. Second, enough working knowledge of failure modes — confident fabrication, stale context, drift after telemetry changes — that analysts know what kinds of wrong to look for. Third, escalation judgment for the ambiguous middle the AI handles worst. Analysts trained only to operate the tool will trust it; analysts trained on its failures will check it.
- Include deliberately wrong AI dispositions in training and grade on catching them
- Teach the failure modes: fabricated detail, stale context, post-change drift
- Drill the ambiguous middle cases — the AI's weakest ground is the analyst's core ground
- Training that never shows the tool failing produces analysts who never doubt it
Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.