How to design agile metrics for my organization?

Almost every organizations keep looking for more and more metrics to control something or especially organizations that I visit for work as agile coach/consultant.  Need for metrics arises during discussion with management but many time management fails to explain objective behind asking so many metrics.

I feel these metrics sometime trigger poor communication, set wrong objective and create absence of trust within team. Organization should focus more on building practices and better innovative processes to meet stated objective. Metrics needed for sure because it helps organization in deciding future course of action and help team to stay focused but metrics without practice? NO because that may demotivate team and demotivated team will produce poor quality product.

Below is an example of metrics without having better practice:- 80% code coverage - What is the reason for this metrics? This is to judge individual performance or ensure code is maintainable? If objective is NOT well communicated to team and practices for clean code is not established then what team member will do?

The team may end up meeting 80% code coverage but how? By adding more test here and there to satisfy metrics but what was the objective? Similarly there are many other metrics like "defect ratio", "defect per feature", "Code complexity", "team velocity", "estimation accuracy" etc. These can be easily manipulated if team doesn’t know objective behind these metrics. There has to be a process for defining metrics and every metrics must have clear objective.

Before measuring ensure there is a practice established. Practice and metric both are aligned to objective. Always better to measure outcome and not output.

Metrics

Organization introduces software development metrics to ensure better quality product and better productivity of development team. Some of the popular metrics listed below:-

• Code coverage
• Defect removal efficiency
• Defects per feature
• Team velocity
• Burndown chart
• Lead Time
• Cycle Time
• Defect reopen ratio
• Test coverage

Look at these closely to understand the purpose and look at what we are doing with these metrics in our organization. Code coverage – This is a metrics to ensure code is well covered by test, code easy to refactor, code is readable and maintainable. Defect removal efficiency (DRE) – Purpose of DRE is to judge productivity in terms of delivery with better/agreed quality. Delivering new feature as well removing defects from developed feature. Similarly others are also linked with either productivity or quality but bigger question “what metrics means and which all to adopt?”

Metrics Adoption

• Metrics should demonstrate effectiveness of organization’s practices.
• Metrics should align with business values
• Metrics need to be linked to project objective to demonstrate achievement.
• The metric needs to be integrated with workflow. Data should get generated automatically rather than manual collection.
• Metrics must be helpful for improvement initiative and preparing action plans.
• Target needs to be defined for metrics and team must be aware of expected lower and upper ranges.

Why Practices?

Practices help in improving quality, team maturity, increase productivity and allow business to forecast releases. Practice mean habit, routine and doesn’t need to remember. It supposed to natural way of software development within organization. These practices should systematically produce data that can be measured to understand effectiveness and efficiency. If practices not in place but only metrics the organization start chasing data and data can be cooked/manipulated that will not reflect reality.

Which Agile Practices?

There are many practices but organization must choose these based on product, expected quality and complexity of the product. Below are some the practices that can be considered. Scrum – Based on agile philosophy that built around inspection, adaption and transparency. Scrum has some prescribed roles, events and artifacts and helps in producing metrics those are align directly with team capacity and helps organization to forecast outcome. There are 3 beautiful metrics that Scrum produces.

• Velocity – It helps organization to know how much a team can deliver per iteration to know release cycle.
• Burndown Chart – It is useful for team to remain focus on commitment and aligned with goal.
• Release Burndown – Helps organization to track deliverables and adjust priority.

Kanban – Useful mainly for tracking continuous changes in production and also in continuous improvement. Again there are 3 key metrics.

• Lead Time - is the time from when a change is submitted by the client (and put on a board) till when all change is implemented.
• Cycle Time - is only the amount of time, that the team spent working on change (without the time that is spent just waiting on the board). Therefore, the Cycle time should start being measured, when change enters the "working" column, not earlier.
• Throughput - is the amount of work items delivered in a given period of time (e.g. week, month, quarter).

TDD – Test driven development is useful practice to reduce quality issues where tests are written prior to code. This automatically helps in writing quality code from beginning. Effectiveness of TDD can be measured using any code coverage tools. TDD also helps in producing less code and only code that needed to pass test. Less code means less defects. This is basically a practice that promote built-in quality rather than afterthought. Similarly there are many more practices that allows to produce variety of metrics such as DevOps, Continuous Integration, Software Configuration Management and Continuous Delivery etc.

Conclusion

Metrics should derived from practices and if not then most likely it will not reflect true picture about productivity, quality and customer satisfaction. All metrics must be aligned with either top line (revenue), bottom line (operation efficiency), CSAT (customer satisfaction) or ESAT (employee satisfaction).