Agile IQ: What we’ve learned about agile delivery from 500+ teams

Good metrics and data is the lifeblood of great executive decision-making. To understand the accuracy of its delivery metrics, ZXM took a science-based approach to the statistical analysis of its Agile IQ® data on delivery effectiveness, cost reduction, risk and how it relates to agile capability maturity. Specifically: What top behaviours drive organisations to higher performance, lower costs, and reduced delivery risk?

The Approach

The statistical methodology

ZXM used a longitudinal multiple-linear regression (orthoganal transformation) to understand the factors that predicted:

  • Reduced costs
  • Improved time to market
  • Improved quality

Variables that we investigated to understand the accuracy of our results were:

  • What is the reliability of the results compared to the assessors (inter-rater reliability)?
  • Were the results different for different industries, e.g. government vs private sector?
  • Were there any questions that weren’t significant contributors to the final data model?

We used experts to do the assessments

ZXM used agile coaches with 5-15 years’ experience to observe teams and Agile Release Trains (agile programs) behaviours. The Agile Coaches’ credentials were supported by industry certifications, including Scrum.org PST and SAFe SPCs.

We didn’t use self-assessments. It produces biassed outcomes.

When measuring effectiveness, self-reporting is heavily influenced by cognitive bias:

  • Self-assessments aren’t accurate. 
  • Self assessment are subjective
  • People over-evaluate their own performance.
  • Self-assessments are time consuming.

How we defined agile behaviour

ZXM used a 6-point Likert scale with 80 behaviours defined from:

  • Agile Manifesto and its 12 Principles
  • Scrum Guide by Jeff Sutherland and Ken Schwaber
  • Lean and Kanban by Daniel Vacanti and Yuval Yeret
  • Management 3.0 by  Jurgen Appelo
  • Extreme Programming (XP) by Kent Beck
  • Systems Thinking by Peter Senge

Assessors were asked whether the they “strongly agreed” that the behaviour occured for the team through to “strongly disagreed”. Either the behaviour occured or it didn’t. There was no “not applicable”.

Who participated?

  • 523 teams over a 10 year period, including software- and non-software teams (HR, finance, marketing).
  • 341 teams were assessed every quarter, some for over a 5+ year period.
  • 23 SAFe® Agile Release Trains.
  • 30 organisations.

The Results

The four behaviours that predict agile outcomes

The results showed that four behaviours were strong predictors of agile delivery outcomes:

Self-Organisation

Self-organising teams are more effective than manager-led teams. They achieved 15-20% more productivity this way.

Agile Values

Those organisations that moved beyond just the symbols of agility and worked to change people's mindsets - the way they think about work - saw stronger behavioural change.

Sprinting

Organisations that had a cadence of rapid, short work cycles, had faster feedback to learn, innovate, and pivot to change.

Continuous learning culture

When people were given room to improve and to be active in learning and taking time out for experimentation, innovation increased causing costs to decrease.

Ágile IQ® is the name we gave to the model that emerged from our analysis.

The model predicts ability to pivot

As agile behaviours strengthenned an organisation’s ability to pivot to adapt to change increased.

agile iq - ability to pivot
Above: Agile IQ model predicts ability to pivot over time

The model predicts cost savings over time

As agile behaviours strengthenned, organisations costs reduced.

agile iq - cost savings 2021
Above: Agile IQ model predicts cost savings over time

The model predicts quality improvements over time

As agile behaviours strengthenned, teams’quality improved, defects and rework decreased, resulting in more time to spend on delivering more. Overall, throughput increased for the same cost with higher quality.

agile iq - quality improvements over time
Above: Quality improvements over time

Conclusions

Google’s Project Aristotle showed that dependability, clear structure and goal clarity produces high performing, effective teams. This supports decades of research from psychology (Loerakker & Van de Grift, 2015, and others) that highlights the importance of self-management over traditional manger-led teams.

Ultimately, teams that embraced these behaviours saw a $2M+ cost savings per quarter over teams that didn’t. 

These four behaviours – self-organisation, agile values, sprinting and continuous learning culture – derived from the Agile Manifesto can help executives create an agile enterprise.

Visit the Agile IQ page to learn more about to assess and monitor your team’s behaviours and get started!

 

About the author

Related Posts

The Ultimate Guide to Agile Reporting

You have reports to deliver to executive. Delivery, risk, and cost is foremost on their mind. What kind of reports on your agile capability and delivery, based on objective agile project management metrics, are going to provide you with the best transparency of what’s really going on?

READ MORE

What Tools Does Your Agile Development Team Use?

The market is full of agile project management metrics tools that help with on-time and on-budget delivery, but where are the tools that help coaches and consultants understand how to scale agile capability development? Do existing tools do the job well enough?

READ MORE

Agile metrics and why team surveys fail

Agile project management metrics often rely on team surveys to find out how agile the organisation is. Team surveys fail for many reasons. Here’s our top tips on what to look out for and how to measure agile in a repeatable and scalable way.

READ MORE

5 agile metrics you won’t hate

Agile project management tends to focus on velocity as a measure of agility. The real things that help leaders measure their agile capability are far more interesting.

READ MORE
search previous next tag category expand menu location phone mail time cart zoom edit close