All Tutorials

Incident Management

WebhookBeam provides a full incident management system that helps you track, communicate, and resolve outages. Incidents can be auto-detected by the health checker or manually created by you.

Auto-Detected vs. Manual Incidents

Type	How It's Created	When to Use
Auto-Detected	Automatically created when a monitored link fails consecutive health checks (reaching the failure threshold).	For straightforward outages detected by monitoring checks.
Manual	Created by you through the dashboard or API.	For planned maintenance, partial outages, or issues that monitoring alone can't detect (e.g. degraded performance, data inconsistencies).

Incident Status Lifecycle

Every incident follows a forward-only status progression:

1. 🔴 Investigating - The incident has been detected or reported. Your team is looking into the issue.
2. 🟠 Identified - The root cause has been identified. A fix is being worked on.
3. 🔵 Monitoring - A fix has been applied. You're monitoring to confirm the issue is resolved.
4. 🟢 Resolved - The incident is fully resolved. All systems are operational.

Note: Status can only move forward - you cannot move from "Identified" back to "Investigating". This ensures a clear, reliable timeline for anyone following the incident.

Severity Levels

Each incident can be assigned a severity level to communicate the impact:

Severity	Meaning	Example
Minor	Small impact; most users are unaffected.	Slightly elevated response times on one endpoint.
Major	Significant impact; many users are affected.	API returning 500 errors for 30% of requests.
Critical	Complete outage; all users are affected.	Website completely unreachable.

Creating a Manual Incident

1. Go to Uptime Monitor → Incidents in the sidebar.
2. Click "Create Incident".
3. Fill in the incident details:
   • Title - A short, descriptive title (e.g. "API Server Unresponsive").
   • Message - An initial status update explaining the situation.
   • Severity - Minor, Major, or Critical.
   • Affected Monitor - Select the primary monitored link.
   • Affected Components - Tag all monitored links affected by this incident.
   • Template - (Optional) Select an incident template to pre-fill the form with prepared messages.
4. Click "Create". The incident is created with status "Investigating".

Posting Timeline Updates

As you investigate and resolve an incident, post timeline updates to keep stakeholders informed:

1. Open the incident from the Incidents page.
2. In the timeline section, click "Post Update".
3. Choose a status for the update:
   • Investigating - Still looking into the issue.
   • Identified - Root cause found.
   • Monitoring - Fix applied, monitoring.
   • Resolved - Fully resolved.
   • Update - General update (doesn't change the incident status).
4. Write your message and submit.

When an update with a new status is posted (e.g. "Identified"), the incident's status automatically advances. Updates are displayed in reverse chronological order on the timeline.

Tagging Affected Components

Incidents can affect multiple monitored links. Tagging affected components helps visitors understand the scope of the outage:

• When creating or editing an incident, select all the monitored links that are affected.
• Affected components will be displayed on the public status page incident timeline.
• You can update the list of affected components at any time before the incident is resolved.

Using Incident Templates

For common scenarios, use incident templates to speed up incident creation:

• Templates provide pre-written messages for each stage of the incident lifecycle (Investigating, Identified, Monitoring, Resolved).
• WebhookBeam includes built-in templates for common scenarios: Server Outage, Database Issues, Network Issues, Third-Party Provider, Deployment Issue, and DNS Issues.
• You can also create your own custom templates.

To manage templates, use the incident templates API endpoint or seed the defaults with the management command.

Writing Post-Mortems

After resolving an incident, you can write a post-mortem report to document what happened and how you'll prevent it in the future:

1. Open the resolved incident.
2. Go to the Post-Mortem section.
3. Fill in the following fields:
   • Title - A descriptive title for the post-mortem (e.g. "Post-Mortem: API Outage on Feb 12").
   • Summary - A brief overview of what happened.
   • Root Cause - What caused the incident.
   • Corrective Actions - Steps taken or planned to prevent recurrence.
4. Save the post-mortem.

Publishing Post-Mortems

Post-mortems are private by default. When you're ready to share them publicly:

• Click "Publish" on the post-mortem.
• The post-mortem will be visible on your public status page alongside the incident.
• You can unpublish at any time if needed.

How Auto-Detection Works

When the health checker detects an outage:

1. The monitored link fails health checks up to the failure threshold.
2. An incident is automatically created with status "Investigating" and severity "Major".
3. An initial timeline entry is posted (e.g. "Health check detected that example.com is down: Connection Error").
4. Down notifications are sent (push and/or email).

When the site recovers:

1. The next health check succeeds.
2. The incident is automatically resolved with a "Resolved" timeline entry.
3. Recovery notifications are sent.