Incident Management Metrics to Track
Key Performance Indicators (KPIs)
To achieve continuous improvement, it is essential to identify the key metrics the team needs to monitor. While these metrics will vary based on your specific needs and priorities, there are several commonly used metrics that serve as industry benchmarks.
These metrics can be grouped into four distinct categories: operational performance, stability, on-call metrics, and throughput.
Operational Performance Metrics
Operational performance reflects how effectively a service meets user expectations, ensuring it is available when needed and performs at its best. The main metric used to measure operational performance is Uptime, which calculates the percentage of time a system remains functional within a specified period, such as a month or a year.
The table below outlines standard uptime goals and their corresponding allowed downtime per year and month:
Uptime
Allowed downtime per year
per month
95 %
18.25 days
1.5 days
99 %
3.65 days
7.2 hours
99.5 %
1.83 days
3.6 hours
99.9%
8.76 hours
10.1 minutes
99.99 %
52.6 minutes
4.23 minutes
99.999 %
5.26 minutes
25.9 seconds
Other metrics include:
- Latency: The time required to process a request or the response delay, both of which should be minimized to ensure an optimal user experience.
- Performance: Typically measured using metrics such as response time, throughput, and error rates to ensure the system operates efficiently.
- Scalability: The system's capacity to handle increased loads without compromising performance or user experience.
Stability Metrics
Stability reflects the system's resilience and its capacity to adapt to changes without triggering compounding failures. The main metrics that help identify issues and understand the system’s behavior post deployment are Change Failure Rate (CFR) and Mean Time to Resolve (MTTR).
- MTTR measures the average time required to resolve an incident.
- CFR quantifies the percentage of changes that lead to failure and is measured as follow: CFR=Failed Deployments/Total Deployments
On-call Metrics
On-call metrics assess the responsiveness and efficiency of the incident management process.
These metrics include:
- Mean Time to Acknowledge (MTTA): measures the average time required to acknowledge an incident.
- Incident Response Time: measures the duration from when an incident is reported to when it is routed to the right team member, including the time taken to acknowledge and provide an initial response.
- On-call Time: measures the time spent on-call to ensure a balanced workload and prevent burnout.
Throughput Metrics
Throughput metrics enable the team to assess the efficiency of the workflow and process within the incident management framework. This helps understand the pace at which changes move through the pipeline and how well the team is managing incidents and alerts.
The main metrics to keep an eye on are:
- Change Lead time: measures the duration from when a change is committed to when it’s live in production, reflecting the efficiency of the deployment process.
- Deployment Frequency: the count of deployments to production over a given time period.
Other metrics to track are the number of incidents and alerts*:
- Number of Incidents: measures the count of incidents in a given timeframe which may reveal trends and patterns enabling proactive incident management.
- Number of alerts: measures the count of alerts in a given timeframe, which helps reduce false positives and alert overload.
* On the difference between incidents and alerts:
IT incidents are events which lead to a disruption or deviation from the regular operating standards of a computer system or network. On the other hand, IT alerts are system notifications to administrators, network operators, incident commanders, or on-call teams that an IT incident has happened or is about to happen, if no action is taken.
Below you find a summary of the key metrics to track:
Once calculated, the following benchmarks can be used to assess performance:
Performance Level
Change Lead time
Deployment Frequency
Uptime
MTTR
Elite
< 1 day
On demand
5%
< 1 hour
High
1 day - 1 week
1 day - 1 week
20%
< 1 day
Medium
1 week - 1 month
1 week - 1 month
10%
< 1 day
Low
1 month - 6 months
1 month - 6 months
40%
Between one month and six months
Source: DORA Accelerate State of DevOps report 2024
Regular analysis of these metrics will provide your team with real-time data to identify recurring issues, bottlenecks, and opportunities to streamline the incident response process, enabling more informed decision-making.
Now that you've identified the key metrics to monitor, it's equally important to gather feedback directly from your team, since feedback loops are crucial for driving continuous improvement in system performance and operational efficiency.
