Improving Program Agility using Evidence Based Measures of Agility in OKRs

OKRs have been an established way to set objectives however the associated activities tend to focus on outputs and not outcomes and impacts. To measure Program Agility and use data to drive improvements, we decided to explore Evidence Based Measures of Agility (adapted from Evidence Based Managment ) when developing OKRs. 

I was working with a well established Product Management Group for a large program of work and their new Product Manager was keen to improve and evolve his group to take them to the next level on their agile journey. We suggested that using Evidence Based Management (EBM) as a framework for OkRs would help shift metrics towards value of the investment to help them make strategic investment decision based on evidence to improve business outcomes.

They were two years into their agile transformation and the entire organsiation was developing enterprise agility.  This program was one of the higher performing programs and had developed to a good standard of competency.


My client was a new Product Manager for the group, and he had noticed the following challenges including:

  • Large batch sizes of the incoming work which led to long turn around for delivery: some pieces of work were estimated to take over 1 Program Increment (PI) to deliver.
  • The bigger the piece of work, the longer the turnaround times for delivery.
  • Longer lead times to deliver a feature from idea to release, increased risks on creating more dependencies to manage within the program and between programs.
  • Potential impact to quality due to delivery pressures to meet stakeholder targets.
  • The stream of incoming work peaked after sprint 3, towards the end of a Program Increment. This put increased pressure on the solution architects and business stakeholders to refine the work and get it “Ready” for teams to start working on.

The Product Manager wanted to understand how to set improvemnet goals and measure the improvements. he was keen to explore options around using Operational Key Results (OKR’s) to create alignment and focus on outcomes, but he didn’t have experience implementing them.

ZXM suggested incorporating Evidence Based Agility Measures (EBM) such as time to market and ability to innovate (pivot) as two key areas to explore when developing OKRs for the group.

ZXM collaborated with the Product Manager and provided guidance on creating OKR’s that would run as an experiment over the Program Increment (PI). The following two OKR’s were the first expereiments we ran to test and address the key challenges above:

  • Reducing batch sizes of incoming work by observing design workshops with the Design SME’s to influence slicing down the work into smaller sizes. This should lead to lower cycle time.
  • Establishing a continuous steady flow of incoming work by actively engaging in PMG for stakeholders to raise work as soon as it comes up.

Developing OKRs as an Evidence Based Canvas to Measure Agility 

A small working group with Product Management was formed and Zen Ex machian developed an EBM Canvas to develop objectives and and the key value measures. The canvas helped identify the problem, the hypotheses we proposed to address the challenge and the key values measures we would use to assess the impact and outcome. This would allow us to test the OKRs as a hypothesis and gather evidence to build in continous learning and improvement. Weekly catch ups were established, progress was discussed and the approach refined.

Key Learnings 

Our key learnings from running OKRs as Evidence Based Measures of Agility  included: 

  • OKR’s created focus and transparency on bottlenecks in the Product Management Group (PMG) process. It drove the Product Manager and team to deep dive into the bottlenecks searching for root cause, data and creating experiments.
  • Increased transparency on quality of data captured was gained. By actually looking at the available data set, we noticed that it was lacking the right level of quality to make problems and impacts measurable. The data quality had been improving over the last 2 PI’s, but we didn’t have a representative data set to make decisions on.
  • We kept the working group deliberately small for this first iteration. Based on the PMG Inspect and Adapt at the end of the PI, we have decided to increase transparency of the OKR’s to the wider PMG community and get broader buy-in from the group including ideas for experiments.
  • We will continue weekly check-ins to ensure progress and finetune experiments.
  • COVID-19 had impacted the initiatives coming into the program of work as teh organisation needed to pivot and adapt to changes in eth marketplace. We therefore need a longer time period to gather more data and refine experiments.

Outcome and Impact

The main outcomes from the first iteration were that the OKR’s developed using the EBM canvas, led to increased focus and transparency on bottlenecks in the PMG process. This ultimately drove continuous improvement in time to market and ability to innovate. Data quality has been improving and we defined further improvement opportunities for the next iteration to continue refining the approach.

The Product Manager shared the data available around the OKR’s in the Inspect and Adapt at the end of the PI. ZXM continue to work with the Product Management Group to capture and to show metrics that assess the agility of the Program. Capturing OKRs based on evidence measures of agility has really helped to improve and address challenges and show progress towards the objectives to the executives. 

About the author

Related Posts:

Increase your Ability to Pivot with Agile Change Management Practices

Traditional Change Management’s linear approach exacerbates the steepness of the change curve and leads to a “Productivity Dip” as it attempts to manage risk through upfront planning. Agile Change Management addresses this dip by iterative customer feedback to focus efforts on the most important activities, determined by customer value and stakeholder impact and outcomes


What Knowledge and Experience do you need to be an Agile Coach?

To be effective, agile coaches require deep, applied expertise and experience across multiple organisations, multiple projects and industries. As the organsiation scales, the Agile Coaching Development Pathway shows the focus of the role as it scales to more responsibilities beyond being a coach of a single team and provides guidance on what knowledge and experience is required to be an Enterprise Coach.


Measuring agile culture with Agile IQ

Agile IQ® is an effective way to measure agile culture. Overall, it’s an effective leading indicator of the changes to mindset, behaviour and culture that’s needed to ensure that your investment in your agile enterprise is on track to deliver the results you need.

search previous next tag category expand menu location phone mail time cart zoom edit close