The Ultimate Guide to Agile Reporting

You have reports to deliver to executives. Delivery, risk, and cost is foremost on their mind. What kind of reports on your agile capability and delivery, based on objective agile project management metrics, are going to provide you with the best transparency of what’s really going on?

The flaws with activity metrics

Agile project management metrics from most tools provide an activity-based view of delivery:

  • How many features were delivered?
  • How many defects were found in production?
  • Team velocity.
  • What was the cost of delivery?
  • Is delivery on-time and on-budget?
agile metrics from state of agile 2021

Many people use these traditional agile project management metrics for reporting. These are great for reporting what has happened. Unfortunately, it doesn’t provide managers with:

  • Links to their OKRs and KPIs, highlighting the impact and outcome of investment that executives have made in specific initiatives.
  • The strength of their agile delivery capability.
  • Why are some agile teams more successful?
  • A view of why some teams are more agile than others.
  • Insights into how to make their agile delivery capability stronger, scalable, and repeatable.

In Twitter, a small discussion came about in Oct 2021, started by GeePaw Hill that agile veteran Ron Jeffries participated in:

A lot of companies do what they call “Scrum” or “Agile” which is actually a thing that should be called “Jira”. Sadly, the Jira methodology is catalytic to any incipient fearfulness it finds in its host. It turns wobbly teams into paralyzed and unhappy teams almost without fail.

To which Ron Jeffries responded:

Jira is an example of a tool that is far easier to use badly than well. as such, I consider it harmful.

When managers focus on agile project management metrics from their tools for reporting , managers only get a picture of whether teams are fully utilised and being efficient, but it won’t tell you:

  • Whether current ‘traditional’ project management behaviours that are in conflict with agile ways of working put future delivery at risk.
  • Whether agile is leading toward reduced costs.
  • How to fix wobbly teams that jeapardise transparent, low risk delivery.

Start with Agile OKRs

Many managers make the mistake of highlighting ‘key results’ as their deliverables. Agile managers set key results as the impact that their deliverables create. This provides the ‘so what’ factor to executives. 

mapping okrs to ebm

Examples of Key Result Measures

Product Value – both new products and existing products:

  • Increase in uptake of the new platform compared to the legacy platform.
  • Usage index improved by 25% compared to 10% last quarter.
  • Data quality has improved by 10%.
  • Completion rates have improved from 20% to 50%.
  • Product cost ratio has improved by 15%.
  • Operational costs have reduced by 25% due to a decrease in defects of 30%.

Capability and business agility:

  • Business agility has increased by 25%.
  • Lead time for changes has decreased by 5%.
  • Innovation rate has improved by 25%.
  • Technical debt has decreased by 50%.
  • Time spent context switching has reduced by 80%.
  • Psychological safety has increased by 8%.

Use value-based metrics against your Features and Epics

Weighted Shortest Job First (WSJF) is a critical part of prioritisation of work by value in Agile Release Trains in the most popular of the scaled agile frameworks, SAFe. Unlike traditional MOSCOW prioritisation methods, Features and Epics are prioritised by their impact and outcome relative to the amount of investment that is required to create those outcomes. This turns traditional requirements and agile project management metrics into investment hypotheses for executives.

 

value-based-prioritisation

WSJF provides managers with the ability to identify the hypotheses that have the potential highest value relative to the size of the investment required.

By using WSJF, executives are:

  • Explicitly investing in an outcome – improving the share price, improving conversion, reducing costs, improving usage, as defined by their OKRs.
  • Determining how much they are willing to finance that outcome.

Reporting to executives must focus on whether or not these Features and Epics are creating the expected outcomes. 

reporting-on-wsjf-features

Use data modelling tools, like Agile IQ to report on your agile capability

Traditional agile project management metrics focus on what has happened. Agile IQ uses data analytics to report on:

  • Forecast cost reduction through supporting teams to reduce re-work through attention to quality, and reducing overtime through supporting a culture and mindset of sustainable pace.
  • Delivery risk profile due to the transparency that agile behaviours, practices, and artefacts create.
  • Strength of the ability to pivot and adapt to disruptive change in the marketplace.
agile-iq-behaviours-and-enterprise-metrics

Because Agile IQ focuses on behaviour and not team-specific metrics like velocity, whole programs and teams’ behaviours can then be easily compared to others of a similar age.

agile iq data model 2021-10-15(sm)

Agile IQ uses a statistical model to measure the strength of behaviours based on industry-standard practices, including the Scaled Agile Framework (SAFe).

The model relates these behaviours to delivery maturity stages, providing robust data analytics for managers to report on:

  • Delivery risk
  • Team effectiveness
  • Improvement actions by your agile coaches
  • Agile leadership
  • Psychological Safety
  • Agile health across all programs

Learn more about agile metrics for managers

Agile IQ® helps managers and executives use data analytics to better report on their business agility

Conclusions

The reporting landscape is complex. Changes to business needs, customer expectations, and techology improvements, all create disruptice change. While activity metrics are important in these environments, they are limited to reporting on what has happened in the past, not whether delivery is at risk in the future based on an understanding of delivery capability.

To report on these key issues, managers need to shift their focus from traditional metrics to outcome an dimpact metrics with tools that use behavioural and statistical models as well as rules engines based on AI and big data.

Agile IQ® provides managers with insights into their business agility and delivery capability so they can report on enterprise outcomes that are truly linked to value.

Learn more about Agile IQ® at: agileiq.com

About the author

Related Posts

What Tools Does Your Agile Development Team Use?

The market is full of agile project management metrics tools that help with on-time and on-budget delivery, but where are the tools that help coaches and consultants understand how to scale agile capability development? Do existing tools do the job well enough?

READ MORE

Agile metrics and why team surveys fail

Agile project management metrics often rely on team surveys to find out how agile the organisation is. Team surveys fail for many reasons. Here’s our top tips on what to look out for and how to measure agile in a repeatable and scalable way.

READ MORE

5 agile metrics you won’t hate

Agile project management tends to focus on velocity as a measure of agility. The real things that help leaders measure their agile capability are far more interesting.

READ MORE

Agile IQ: What we’ve learned about agile delivery from 500+ teams

What top behaviours drive organisations to higher performance, lower costs, and reduced delivery risk? To understand agile delivery metrics, ZXM took a science-based approach to the statistical analysis of its Agile IQ® data on delivery effectiveness, cost reduction, risk and how it relates to agile capability maturity.

READ MORE
search previous next tag category expand menu location phone mail time cart zoom edit close