I have been working on scaled agile projects for the past 8 years and a couple of years ago, I asked by an executive how he would know if the money they invested on setting up these agile release trains was working? He was essentially asking me to prove the “ROI” for agile and to measure delivery of value.
Numerous industry surveys have been conducted over the last decade that report a number of outcomes from being agile, including an improved ability to manage changing priorities, increased team productivity, improved transparency, improved team morale, and a faster delivery. Unfortunately, the collection and analysis of empirical evidence of these outcomes is rarely conducted.
Without robust empirical measures, organisations tend to turn to the symbols they can see, such as team happiness, velocity, throughput, visualisation of work by using post-it notes, attendance at Agile events, or number of PI planning events held. While these outward symbols have their uses, they don’t tell the real story of agility at scale and don’t talk to the executive audience that needs to know about performance in market share, share price and speed to market.
The lack of a standard and evidence-based mechanism to measure performance and agility leaves many organisations in the dark regarding what activities are needed to not only lead to agility, but what activities leaders should support and promote help to make enterprise agility sustainable, repeatable and scalable. This is where Agile OKRs come in.
Agile OKRs
Traditional project management Objectives and Key Results (OKRs) have been widely used in the industry as a framework for reporting. They represent ways to establish goals and report on their progress. The objective sets out what we want to achieve and the key result areas that deliver it. For me they don’t go far enough to provide the “so what, now what?” factor that executives want answered to make data driven decisions for the organisation. velocity means little to the Board room and shareholders.
Traditional OKRs using SMART objectives are Ok as a starting point and better than not having any goals and objectives, but they miss out on the most vital element of strategic planning – the outcome and impact. “Progress measures” like milestones completed or tasks completed measure activity which just tells us where time was spent, not whether value was produced. Using progress as your main guide can be misleading and misalign business and delivery organisations. Likewise, “Artefacts produced” tells us about Output; it tells us what got created, but not whether we are adding more value to the organisation. As such, OKRs built around key results areas based on activity and output, don’t give executive a view of whether investment initiatives are of value or not.
Agile OKRs Measure Flow and Improvement in Value
Agile OKRs instead focus on Key Value Areas to measure value, outcome and impact derived from product and service delivery rather than just progress on activities. Key concerns for business agility are improvement in outcomes, reducing risks and optimising investments. Developing Agile OKRs based on Evidence Based Management (EBM) approach, helps establish a starting point to measure improvement in flow of value to the market and the organisation capability need to deliver value.
Flow of Value Metrics
The key value areas to focus on for Flow Metrics are Ability to Innovate and Time to market. These outcomes driven metrics show the improvement or decrease, in your capability to provide products and service to customers.
Ability to Innovate
The focus here is on understanding and measuring how quickly can we deliver new capability to serve customer needs. It also asks what the is quality like? What prevents us from delivering new value? Anything that prevents users from benefiting from the innovation will reduce your ability to innovate therefore the focus should be on measuring impact through flow metrics such as:
- Production incident trends
- Time spent context switching
- Downtime trends
- Continuous improvement
- Average time onboarding
- Innovation rate
- On Product index
Time to Market
The ability to adapt/pivot quickly to deliver new products is a key factor in a dynamic market environment. Looking at time to market flow metrics helps organisations understand the amount of time it takes to deliver value from ideation to deliver. Many things can reduce time to market such as bottlenecks in the value steam, technical issues, handovers and maintainability of products and technical debt. Therefore, measure flow through the value pipeline is critical. leading indicators of successful outcomes are:
- Frequency of build success
- Release stabilisation
- MTTR (mean time to Repair)
- Reduced handovers
- Red tape reduction
- Throughput
- Cycle time
Improvement in Value
Current Value
How do we measure if we have maximised the value delivered to customers and stakeholders? Look at outcome measures that are leading indicators and whether they are increasing or decreasing such as:
- Customer satisfaction gap
- Usage index
- % retention of subscribers
- Average session time
- Daily active uses
Unrealised Value
Unrealised value looks at whether additional value can be created in the marketplace. Is it worth the effort and risk to investment to pursue these opportunities? Key value measure here looks at trends in:
- Customer acquisition or detention
- % of new vs existing customers
- Market growth/decline
Agile OKRs start with Strategic Outcomes
Agile OKRs link the Company Vision and Strategic Outcomes to the Product Strategy and value of the solution delivered. They ensure transparency and traceability for each Feature and Backlog Item to the purpose and vision of the organisation.
Everything the Solution or Release Train are working on, should link directly to the business strategy and achieve this through focus on improving capability to deliver value by working on understanding customers and keeping them satisfied, increasing ability to innovate and decrease time it takes to get new products or features to market. It is critical to link strategy outcome to the work to show why investment in this area will lead to the desired impact for the organisation and its customers.
This is a useful template to build your Agile OKrs. Describe the outcome as a hypothesis. Importantly, at its strategic level, it represents an outcome that is worth investing in.
We believe [this initiative] for [this customer] will achieve [this outcome]. We know that this will be true when we see [these leading/lagging indicators] change.
The domain of Agile Product Management, looks at the holistic picture of what it takes from ideation to delivering value to customers. Vision, solution delivery and validation are a constant that ensures this line of sight from vision, business strategy and product vision to product strategy, solution discovery, delivery and value. Identifying, visualising, and optimising value streams is the primary method a Lean enterprise employs to shorten time to market while improving the timeliness, quality, and value of its products and services.
Unlocking Business Success
Agile IQ®'s flow metrics benchmarking gives your OKRs the insights you need to drive the flow of value in your organisation Learn how.
Understand Flow of Value through the Value Stream
Understanding the value stream and how to promote Solution Train and Agile Release Train flow is vital. Every product or service has an Operational Value Stream (OVS), to deliver a specific product or service to the customer. Each value stream represents the sequence of steps an enterprise uses to deliver value to its customer and highlights the flow of value, delays, rework, and bottlenecks. They illustrate how the current flow affects the people who do all the work.
How quickly can we go from ideation to putting the product into the customer’s hands and getting feedback? What is stopping us from being faster? Can we pivot quickly if we need to? Understanding the value stream’s purpose and attributes and how the enterprise serves its customers, allows us to look at ways to optimise the operational value stream. This is why measuring metrics such as Ability to Innovate and Time to Market are where I start to build my Agile OKRs.
To ensure they focus on outcomes and impact, when building Agile OKRs I start with the strategy and problem we are solving and then ask the “how will we know we have succeeded” question. Did this attribute we are measuring, increased or decrease and by what percentage? This is why measuring metrics such as Ability to Innovate and Time to Market are where I start to build my Agile OKRs.
