Agile metrics for your enterprise: OKRs are out. EBM is in.

OKRs – Objectives and Key Results – have been used quite widely in the industry as a framework for reporting. Created by Intel and adopted by companies like Google, Twitter, LinkedIn, Oracle and others, it represents one way to establish goals and report on their progress. Unfortunately, it’s fundamentally flawed.

OKRs measure the wrong thing

An ORK objective sets out what we want to achieve – e.g. improve annual budgeting and business planning and the key result areas that deliver it. The question is, though, “so what?” Why? OKRs miss out on the most vital element of strategic planning – the outcome. As such, their measures don’t give executive a view of whether investment initiatives are of value or not. They only measure activity and reflect “busy-ness”.

An example OKR from Weekdone.com

Evidence Based Management (EBM) and Key Value Areas

What is EBM?

Evidence-Based Management (EBM) is a framework organisations can use to help them measure, manage, and increase the value they derive from their product delivery. EBM focuses on improving outcomes, reducing risks, and optimising investments. In particular, it is very well suited to supporting business agility. 

How does EBM differ from OKRs

Where people use OKRs to measure progress of explicit activities and deliverables, EBM’s focus is on value, outcome and impacts.

Start with the outcome, not the objective

EBM’s framework establishes the starting point as improving value in the market or within organisational capability. EBM has four Key Value Areas (KVA) to describe the outcome:

  • Improving current value (CV) – Value being delivered to customers/users today.
  • Leveraging and accessing unrealised value (UV) – Value that could be realised by identifying and meeting potential needs.
  • Improving the organisation’s ability to innovate (A2I) – Ability to deliver a new capability that might better serve a customer need.
  • Improving the organisation’s time to market (or delivery time) (T2M) – The ability to quickly deliver new capability, service or product.
zxm-ebm-evidence-based-management-2020

The outcome is described as a hypothesis. Importantly, at its strategic level, it represents an outcome that is worth investing in.

 

We believe [this initiative] for [this customer] will achieve [this outcome]. We know that this will be true when we see [these leading/lagging indicators] change. 

What are leading and lagging indicators?

A leading indicator is a predictive measurement, whereas a lagging indicator is an output measurement. The difference between the two is a leading indicator will indicate early signs of progress toward the the Lagging Indicator. EBM uses both, as it recognises that strategic investment needs early signs that progress is being made toward an outcome – improvement in customer satisfaction, market share, or overall market growth.

Current Value (CV)

The goal of CV is to maximise the value that an organisation delivers to customers and
stakeholders at the present time; it considers only what exists right now, not the value that might
exist in the future. 

While project management tradition would have us report on time and on budget, CV would suggest reporting on features actually used by customers as true success of strategic initiatives equals maximising value.

Key Value Measures

Leading Indicators

Investor/stakeholder satisfaction

Form of sentiment analysis to gauge stakeholder engagement and investment satisfaction.

Employee satisfaction

Form of sentiment analysis to gauge employee engagement.​

Usage index

Infers the degree to which customers find the product useful.

Customer satisfaction

Helps gauge customer engagement and happiness with the product.​

Subscriptions

Form of sentiment analysis to gauge stakeholder engagement.

Daily active users

Infers the degree to which customers can engage with product information.

Ratio of leads captured

Form of sentiment analysis to gauge willingness to engage,

Ratio of touchpoints with clients

Helps gauge customer engagement.

Lagging Indicators

Product cost ratio

Total expenses and costs for products including operational costs compared to revenue.

Employee turnover

Turnover gives wider insight into your company culture and the health of your employer brand.

Revenue per employee

Ratio of gross revenue per number of employees as a competitive indicator.

Annual sales

Ratio of growth in sales for existing products

Net Promote Score

A gauge of the loyalty of customer relationships.

Cost of customer acquisition or defection

The cost combination of how much value having each customer typically brings to the business.

Revenue retention

Total revenue minus any revenue churn (caused by departing customers, or customers who have downgraded) plus any revenue expansion from upgrades, cross-sells or upsells.

Conversion rate

The number of visitors to a website that complete a desired goal (a conversion) out of the total number of visitors

Unrealised Value (UV)

Can any additional value be created in our market or other markets? Is it worth the effort and risk to pursue these untapped opportunities? Should further investment be made to capture additional unrealised value? All these questions are asked by executives using EBM.

Key Value Measures

Leading Indicators

Competitor strengths / weaknesses

Customer acquisition or defection

Lagging Indicators

Market share trends

The relative percentage of the market controlled by the product.

Overall market growth/decline

Relative to market share trends.

Ability to Innovate (A2I)

The goal of looking at the A2I is to maximise the ability to deliver new capabilities and innovative solutions. Executives should continually re-evaluate their A2I by asking:

  • What prevents us from delivering new value?
  • What prevents customers or users from benefiting from that innovation?

As low-value features accumulate in digital products, more budget and time is consumed maintaining the product, not increasing capacity to innovate. Anything that prevents users from benefiting from innovation, such as difficult to install software, low usability, or even a lack of specific capabilities, will also reduce A2I.

Key Value Measures

Leading Indicators

bug

Production incident trends

Time spent context switching

Technical debt

Architectural coupling

New experiments tried

Revenue driven by use of new technology

Downtime trends

Velocity trends

Sprinting

Continuous improvement

Servant Leadership

Agile values enacted

Lagging Indicators

Innovation rate

Usage index

Culture of innovation

Agile culture /
Agile IQ

Time to Market (T2M)

The goal of looking at Time-to-Market is to minimize the amount of time it takes for the organisation to deliver value. Without actively managing Time-to-Market, the ability to sustainably
deliver value in the future is unknown.

Questions that executive need to continually re-evaluate
for time to market are:

  • How fast can the organisation learn from new experiments?
  • How fast can teams learn from new information and adapt?
  • How fast can the program deliver new value to customers?

A variety of things can reduce the Time-to-Market: everything from removing internal communication bottlenecks to improving delivery pipeline automation to improving application maintainability and removing technical debt; anything that reduces time spent waiting or time spent performing work.

Key Value Areas

Leading Indicators

Frequency of build success

Release stabilisation trends

Build pass/fail trends

Mean time to repair

Time spent context switching

Red tape reduction

Reduced handovers

Reduced non-value adding tasks

Sprinting

Continuous improvement

Servant Leadership

Agile values enacted

Reduced queue lengths

Reduced batch sizes

Reduced overburdening the system

Reduced unevenness,
non-uniformity

Lagging Indicators

Lead time

Time to learn

Cycle time

Release frequency

Conclusions

Where OKRs fail to provide an holistic perspective of product management and delivery performance, EBM’s Key Value Areas provide a balanced perspective of both customer-facing and internal-facing factors that support improvement in the face of ubiquitous, disruptive change.

EBM is a timely reminder that profitability and market survival in the 21st century isn’t dependant on just current value, but also unrealised value, and internal ability to improve, innovate and react to changing needs in an agile way.

Reporting and tracking with EBM Canvas

ZXM’s EBM canvas aligns strategic planning activities with investment initiatives and Evidence Based Management (EBM). It fits in perfectly with Toyota Kata planning and learning cycles.

Related Posts:

Agile Strategic Planning for Executives

Can you predict the future? Can it really be planned in the 21st century? In the face of relentless technological change, disruptive forces in industry after industry, global competition, and so on, strategic planning often seems like pointless wishful thinking.

READ MORE

Agile Leadership

What does it take to be an agile leader? 21st century leaders face a raft of challenges: technological change, disruptive forces in industry after industry,

READ MORE

Agile Strategic Planning

Can you predict the future? Can it be planned in the 21st century?In the face of relentless technological change, disruptive forces in industry after industry,

READ MORE

Copyright © Zen Ex Machina® and ™ (2019) All rights reserved. ABN 93 153 194 220

search previous next tag category expand menu location phone mail time cart zoom edit close