When it comes to agile project management metrics, many product managers turn to understanding whether delivery is on-time and on-budget. Others turn to velocity.
While metrics are a great way to provide a window into the state of delivery, these types of activity-based metrics fail executives on a number of levels:
- It only tells you what has happened, not whether delivery is likely to fail in the future.
- Budget spent on producing features doesn’t equal whether the investment has made the difference that was intended in the project management business case. You could be spending money in entirely the wrong place.
- Velocity is highly variable and changes over time. It’s a poor predictor of future work and productivity. If you reward greater productivity and base it on velocity it’s too easy for teams to game the result and just increase their Story Point estimates.
These types of traditional project management metrics ultimately fail to provide Product Owners and Product Managers with a sense of what their overall agile capability looks like, whether its strengthening, and what kinds of returns on your investment is agile providing you and the enterprise. What Product Owners and Product Managers need from metrics is a view of whether teams are effective or not.
1. Clear Structure
According to Google’s Project Aristotle, structure and clarity are key to team effectiveness. In effective teams, people need a strong understanding of:
- Their role versus their job title. Are they both a “Developer” and the capability lead for testing?
- Other roles in the team. What are the expectations of the Scrum Master and Product Owner? Are all members of the team aware of what the role is there to achieve?
- The guardrails for delivery. Have managers set expectations regarding the minimum set of practices expected for all teams and team members? Is the expectation that all teams use Scrum as their minimum set of activities to apply empiricism?
2. Goal Clarity
Goals are part of the team effectiveness equation, especially Sprint Goals and the Product Goal. Without measurable goals, Sprints devolve into feature factories which impact a team’s effectiveness.
Using agile OKR metrics and measures supported with Evidence Based Management (EBM) is an effective way to create a clear structure and set expectations about team performance.
3. Batch size
The “Small Batches Principle” is a fairly well established practice in Lean and manufacturing:
"The small batches principle: it is better to do work in small batches than big leaps. Small batches permit us to deliver results faster, with higher quality and less stress."
In product development, the benefits of small batches are immediate. You get less waste since you get feedback quickly and can adjust. There is less risk of doing a lot of work that is useless.
Don Reinertsen highlights that while small batch sizes increase flow, it’s also important to consider the transaction costs versus holding costs. He emphasises that an economic trade-off has to be made in product management and explains how this applies in particular to software engineering.
4. Dependability
Team effectiveness is also dependent on whether its members can reliably complete quality work on time (by the end of the Sprint and in accordance with the Definition of Done) or if they shirk their responsibilities.
Many teams claim “we deliver”, yet fail when it comes to:
- Having a clear Definition of Done that outlines the standard of quality for the team.
- Producing an Increment at the end of every Sprint.
- Inspecting progress toward their Sprint Goal every day at the Daily Scrum.
- Being cross-functional – not allowing their functional roles and titles get in the way of supporting their team to achieve their Sprint Goal.
- Promoting transparency of the status of work through engaging stakeholders at Sprint Review to inspect their work.
5. Mastery
Mastery is the desire to improve. When people are motivated by mastery, they tend to see their potential as being unlimited, and constantly seek to improve their skills through learning and practice. Those people who seek mastery tend to want to attain it for its own sake.
In his 2009 book, “Drive”, Dan Pink describes research undertaken by psychologists and professors in MIT from 1971 to 2017. The research shows that monetary rewards can fail to improve people’s engagement with tasks, and may even damage it. In contrast, when workplaces provide the room for people to be intrinsically motivated through rewarding creativity and innovation, their teams thrive, become more motivated, and tend to be happier.
Measuring these aspects of team effectiveness
When we ran statistical models of what reliably and repeatedly helps makes a team agile, all these behaviours correlated to team effectiveness arose. Importantly, the stronger these behaviours the greater the outcomes such as:
- Decrease in time to pivot.
- Decreased delivery cost.
- Improvement in quality.
The statistical model for assessing the strength of these behaviours is called ‘Agile IQ‘.
Conclusions
You can’t improve what you can’t measure. Unfortunately, many agile project management metrics are based solely around activity metrics and efficiency. While these metrics are useful, they don’t:
- Measure the strengths of the behaviours that create strong agility.
- Tell leaders why some agile teams are more successful than others.
- Identify what makes teams agile.
- Recommend actions to make agile success repeatable and scalable.
Human behaviour is complex and so it needs predictive analytics, statistical models, and rules engines, not traditional descriptive metrics.
Agile IQ® is a robust statistical model and tool that empowers teams and managers to understand the strength of their behaviours and promote the metrics that really measure agility.
Learn more about agile metrics
Agile IQ® helps leaders measure their team's agility with agile metrics based on statistical models of industry standard practice.
