Course contentsShow
← Cloud and Infrastructure
Lesson 2133 of 2,98945. Observability - Dashboards and SLOsFree lesson

Incident Command System

Implement structured incident command with defined roles: commander, communications lead, and operations lead.

Incident Command System

What you'll learn: How to organize chaos during a major incident using clearly defined roles that prevent confusion and accelerate resolution.

The Core Problem

When a critical outage hits, everyone wants to help—but without structure, multiple people issue conflicting commands, duplicate work happens, and nobody knows who's actually in charge. The Incident Command System (ICS) solves this by establishing a clear hierarchy with three core roles.

The Three Critical Roles

1. Incident Commander

The single decision-maker who owns the incident. They delegate tasks, make judgment calls on trade-offs, and decide when the incident is resolved. Think of them as the conductor of an orchestra—not playing every instrument, but ensuring everyone plays in harmony.

Responsibilities: Declare severity, assign roles, make final decisions, approve communications.

2. Communications Lead

Manages all stakeholder updates: status pages, internal notifications, executive briefings. They shield the technical team from constant "what's the status?" interruptions.

Responsibilities: Post to status pages, coordinate with support teams, prepare incident summaries, handle executive communications.

3. Operations Lead

Coordinates the hands-on technical work. They direct engineers investigating root causes, implementing fixes, and validating solutions. The Incident Commander tells them what needs fixing; they figure out how.

Responsibilities: Assign technical tasks, track mitigation progress, coordinate with on-call engineers, manage runbook execution.

Why This Structure Works

Without ICS, five engineers might simultaneously restart the same service while nobody updates customers. With ICS, the Incident Commander says "Operations Lead: restore service; Comms Lead: notify users"—clear ownership, no overlap, faster resolution.

Key Takeaway: The Incident Command System prevents chaos during outages by assigning exactly three roles—Commander (decides), Communications (updates), and Operations (executes)—ensuring clear accountability and efficient resolution.