There is a question I get asked a lot during my individual and also team coaching sessions. The million-dollar question that many of us have heard asked before, whenever one speaks of agile transformation is below.
How will we know if the agile transformation has been successful?
For this article, I will be narrowing down the scope of agile transformation into scrum implementation because most of the time, we associate agile with scrum. There's nothing wrong with that, as scrum methodology is the most commonly practiced one within the agile methodology toolkit.
So, how would one determine if Scrum has been implemented successfully? Below are my top three recommendations for organizations and teams who are interested to learn how to go about measuring scrum.
Before anything can be measured, seek to first understand the business.
What is the business model that your organization runs on? If there are several, which one does your team fall under? How fast is your organization in taking an idea to production? Does your organization need or want it to be fast? How frequent are we looking at? One week? One month? Few months?
By slowly immersing yourself in the team and also the organization, you will soon understand the nature of the business. For instance, how often do customers request new features? What is the average customer lifetime value? How about the average customer acquisition cost? What is the Net Promoter Score for the product? Is it above or below 70?
The above are example questions that are important to understand the business nature. If your business is measuring some of the above, by analyzing the data, you will be able to predict how adaptive your scrum teams need to get in order to impact the business significantly.
In a nutshell, what matters most to your business? From that conversation, your scrum team will be able to position themselves to see how they can contribute to business impact.
The next section will describe the thought process of defining metrics to be measured.
Identify what impact a scrum team can bring, and measure that impact.
In my opinion, a scrum team is able to impact the business in three key ways.
- Time to Market: This is about the ability of an organization to reach its customers. Example metrics would include cycle time. What is the average time for the scrum team to release an idea to production?
Release frequency is also a good indicator to observe how frequently products are being released to remain competitive. Another worthwhile metric to consider is releasing stabilization period, which shows the amount of time spent correcting problems between the point developers say it is good to go live and the point where it is actually released to the customers.
This stabilization period may give information on which part of the development practices require improvements. Time-to-learn is also useful to gather customer insights after releasing the product to them.
- Ability to Innovate: Technical debt is a great indicator of a scrum team's ability to innovate on an idea. The larger the technical debt, the more unlikely the team is able to take an idea to production, let alone innovate because of 'quick and dirty' work that requires remediation afterward.
Measuring escaped defects will also help the scrum team to visualize how frequently defects are being caught by users in production. Ideally, no defects should reach end-users in production and they should be detected as early as possible during the development process.
What would be a greater stretch goal is to see how the scrum team can together prevent defects from occurring, instead of reacting to defects that have already been produced. Time spent context switching may also give simple insights on how frequently a person needs to divert his/her focus from building an actual working software to attending meetings or interrupting to help people outside of the team.
- The Existence of the Scrum Team Itself: Without a Scrum team, there is minimal-to-none business impact that can be made. The cross-functionality of a scrum team is what exactly powers the business, they are a team of individuals who can build a product from an idea to customers saying thank-yous.
To get started on measuring how the scrum team is doing, I would recommend getting feedback from the team either directly or indirectly via Net Promoter Score (NPS), team satisfaction surveys, team turnover rate, utilizing daily stand-ups alongside sprint retrospectives and a tool that I personally enjoyed experimenting with, called the Niko Niko Calendar.
Below is what the calendar looks like. I am more than willing to elaborate more about the Niko Niko Calendar if this catches sufficient interest and attention.
Review frequently and enlist actions to stay on track.
Once the metrics have been identified for measuring and tracking the progress of a scrum implementation, they are not valuable unless they are reviewed and acted upon. So, my recommendation is to review the metrics on a weekly/monthly/quarterly basis (review frequency may vary depending on the metrics that have been set) to get a sense of how things are progressing in your organization.
Do not get frustrated or disappointed if the train derails every once in a while, it is perfectly normal for such scenarios to happen! What's next is to anchor onto the vision, and determine action items to stay or get back on track.
As a footnote, perhaps ask yourselves this question instead as food for thought.
What would tell you that an agile transformation has been successful?
