Flow metrics are critical to shedding light on the effectiveness of your organisation. But what are flow metrics and why should you make the change from project management metrics like time, scope and risk, to impact and outcome by using insights from flow metrics?
Project management metrics
Every executive is familiar with traditional project management metrics [1]:
- Scope: The features of a solution.
- Time: How long it will take to deliver the solution, from when the project starts to when the project concludes, after which the solution is handed to the customer to then realise its benefits.
- Resources, such as cost: The overall budget spend required to deliver the scope within the time available.
This triple constraint, as depicted by the Iron Triangle of the 1950s is a reminder to business stakeholders that in delivery of the project they have options for “good, fast, cheap. [So] choose two.” This is often expressed as the Common Law of Business Balance: “you get what you pay for”.
Pizza, DoorDash, and project management metrics
Using triple constraint metrics are useful, but ultimately only insofar as they answer the question “when will I get it?”. We ask this type of question every time we order a pizza or ask DoorDash to facilitate door-to-door food delivery. We make our order, we’re told how much it costs, and then we get status of when we’re likely to have dinner in our hands!
Above: Tracking delivery time with Domino’s pizza app. Source: @marisabattistel https://twitter.com/marisabattistel/status/821253611984285696
Time is a great metric for setting expectations about how soon the driver will arrive and we can fulfil our hunger needs. It’s perfect for simple things we can estimate fairly accurately, like a linear manufacturing process. It doesn’t work out so well, though, when the environment is more complex. This is where “on time” and “on budget” metrics are often misused. They define success as delivering the required scope, like a pizza, but are silent on the complexity and dimensions of business success:
- The impact on stakeholders.
- The benefits for customers.
- The desired outcomes for the organisation.
Project management metrics simply don’t tell us whether a project will create its intended outcomes. If we’re using traditional project management methods, including Waterfall, we will never know whether the project will have the desired effect until the very end – when it’s used by our customers. And with traditional project management methods, many years could pass before we will know if we’ve delivered value. In a world of rapid and constant disruptive change, this delivery method is not a viable operating model or company strategy.
Impact metrics measure value
Modern executives need a clear line of sight from idea and investment to delivery and value. This requires a shift from project management metrics that focus on activities to measuring impact and outcome.
Above: The shift in focus from project management metrics to value-based metrics
That’s not to say that reporting on budget spend and progress of delivery isn’t important. All product managers have to account for how they spend the company’s limited resources. The focus, however, is not “what do I get”, but “how much does the company needs to spend to make a difference”, i.e., how does the organisation know it’s making an impact so we ensure that customers will get the outcomes they require.
With traditional project management methods, many years could pass before we will know if we’ve delivered value. In a world of rapid and constant disruptive change, this delivery method is not a viable operating model or company strategy.
The Evidence Based Management (EBM) framework [4] provides executives with a more holistic view of value – from idea through to investment, delivery, and measurement of the impact our solutions have made with customers.
Above: EBM Framework™ by Scrum.org®
Start with Usage Index
Executives often first turn to Usage Index as their first measure of value. We can be on-time, on-budget, and deliver the required scope, but are features being used by customers? Are the new services we’re delivering being used? Usage Index is an assessment of viability and is used to:
- Understand whether product ideas meet expectations.
- Deliver insights to executives as to whether they’ve met a customer satisfaction gap.
- Help executives to decide on whether to continue to spend money to support an existing product or service, or to pivot to spending money elsewhere where it will make a bigger impact.
Product Cost Ratio
What is the overall relationship between the relative costs and benefits of an initiative? The product cost ratio calculates the total expenses and costs for the product(s)/system(s) being measured, including operational costs compared to revenue.
- If an initiative has a product cost ratio greater than 1.0, the investment is expected to deliver a positive net present value to an organisation and its stakeholders.
- If an initiative’s product cost ratio is less than 1.0, the investment costs are likely to outweigh the benefits, and should likely be abandoned or phased out in preference of a more effective solution.
Customer Satisfaction Gap
Customers have become increasingly demanding. As technology changes more rapidly the choice of different services expands options for customers. They’re more likely to walk away faster, complain on social media, or write to their local government representative, if engagement requires more effort than necessary. The customer satisfaction gap refers to the difference between their expectations and perceptions.
There are five key gaps in customer satisfaction [5]:
- Gap 1: The Knowledge Gap: Executives don’t not fully understand what the customers really want and so invest in the wrong solution.
- Gap 2: The Policy Gap: Executive understand what customers want but aren’t able to fully translate these needs into solution definitions that make an impact.
- Gap 3: The Delivery Gap: Executive understand what customers want but the people who “produce” the service are unable, unwilling, or just do not know how to fully reach the specified and needed standard of that service.
- Gap 4: The Communication Gap: Expectations can be formed by customers based on corporate communication that are only partially met or not met at all.
- Gap 5: The Customer Gap: This gap typically arises when the customer sees a characteristic of service quality as undesirable. This gap is also now used to identify overall customer dissatisfaction that appears as the result of the sum of all the other gaps.
Which value metric should you use?
Value isn’t measured with a single metric. It’s crucial to understand impact and outcome from a wide range of perspectives, so executives need a selection that creates diversity without a significant overhead of collecting and interpreting the data.
5-steps to implementing value metrics:
- Pick both leading indicators that will provide the earliest indication that solutions are of value to customers, such as usage index.
- Pick some lagging indicators, such as customer satisfaction gap.
- Iteratively and incrementally deliver solutions.
- Inspect the metrics every month. If you’re using an agile framework, inspect the metrics at the end of every Sprint (“Sprint Review”) and if your metrics aren’t moving, consider selecting different metrics.
- If metrics haven’t moved for a while, it’s likely because the solution isn’t having the desired impact. It could then be time to pivot and change to a new solution.
In this way, value metrics have a strong, close tie with OKRs. Together, these metrics ensure that targets and objectives are based on managing the product, not on managing people.
Flow metrics are all about speed to value (without compromising quality)
Flow metrics provide a superior view of how effective an organisation is at delivering value [2]. Key flow metrics for executive and product management teams include:
- Lead time / Time to Market: The time from idea through to putting value in the hands of customers.
- Throughput: The number of items “finished” per unit of time, e.g., per Sprint.
- Work in progress (WIP): Provides transparency into progress towards improving their flow.
Above: Agile IQ® dashboard with flow metrics and industry benchmarking
Inspecting these metrics provides insights into the level of capability of the organisation to deliver value effectively and its ability to pivot when investments are not having the desired impact. Agile Release Trains inspect these metrics at Inspect & Adapt workshops to enable executives to invest in capability building to improve the flow of value on a quarterly basis. Scrum teams inspect these metrics at Retrospectives in order to determine what improvement actions should be initiated or refined the very next Sprint to improve the flow of value.
Revolutionise Your Value Stream
Agile IQ®'s flow metrics benchmarking can help your organisation succeed. Learn how.
Conclusions
Ability to pivot rapidly is key in a world of constant change. Customer needs evolve, technology disrupts the market, while our traditional bureaucracies are slow to make decisions about investments and project budgets. Project management metrics are important, but they don’t inform executives on the dimensions of business success or how quickly and efficiently we can deliver value.
Armed with flow metrics and tools like Agile IQ®, executives can finally make informed decisions on what it takes to deliver value, and give us our best gauge of true business agility.
References
1. PRINCE2 (2019) Project management triangle: overview of the triple constraints. Online at: prince2.com/aus/blog/project-triangle-constraints
2. Scaled Agile Inc. (2023). Metrics. Online at: scaledagileframework.com/metrics/
3. Yeret, Y. (2018) 4 Key Flow Metrics and how to use them in Scrum’s events. Online at: scrum.org/resources/blog/4-key-flow-metrics-and-how-use-them-scrums-events
4. Scrum.org (2023) Evidence-Based Management™ (EBM). Measuring value to enable improvement and agility Online at: scrum.org/resources/evidence-based-management
5. Wolf, P. (2023) How to Run a Gap Analysis for Customer Service & Understand Customer Satisfaction. Online at: custify.com/blog/gap-analysis-for-customer-service/
