Metrics That Matter - Part 3 - Change Success Rate
In my previous Metrics that Matter posts, I discussed how high performing IT organizations use mean time to repair (MTTR) and first fix rate (FFR).
In this post, I'll excerpt from my recent article that covered another metric that stratified the high performers from medium and low performers that we have studied in the IT Controls Performance Study.
This key metric is change success rate. The key finding from the study is that top performers have a more stringent definition of change success rate, AND have higher metrics.
A common measure of of change success rate includes the number of changes successfully deployed without creating an incident, service impairment or disruption of work. However, top performers we studied focus on process as well as outcomes, and also include process exceptions as part of their definition of success. To them, a change doesn’t necessarily have to cause an incident to be defined as unsuccessful.
For instance, when a change does not go accordingly to plan would also count as a process exception. This could be such as when a change exceeds the scheduled implementation time or varied from the deployment plan, requiring undocumented steps. High performers would count all these examples as failed changes. (We’ve observed that high performers use the term "change process exception" to define this condition, to distinguish this from a failed change.)
For a closer look at measures and variance of top, medium, and low performers, please reference this article. Link to full article.