Issues with Integration as you Scale? – Agile Principles and Practices to Mitigate

We were working on a large scaled program of work that involved over 40 delivery teams needing to coordinate and work together to deliver the end-to-end solution. Many teams were component teams aligned to architecture and platform and this meant they experienced delays and impediments to releasing more frequently due to inter-train dependencies. They were moving to more agile ways of working and wanted to move from a “big bang” once a year release to more frequent releases and ultimately a continuous integration approach (DevOps) where a lot of the manual integration would be automated and done by the agile teams themselves. We had started working with this program in the initial stages of their agile transformation about 12 months previously. The leadership knew this would be an iterative, incremental journey and would take potentially a number of years to develop the type of culture and change they had envisaged. Whilst cross-functional teams are the cornerstone of Agile, at scale, it sometimes takes a very long time for organisational agility to evolve. Consistent challenges that prevented agile teams from delivering features end-to-end within a Program Increment included:

• Organisational Structure: Teams were mostly  back-end and relied on another team for other  components of software,  front end enhancements, database layer, integration and API
• Immature Agility: teams were using Scrum, but many of the other teams they were working with were using more traditional approaches, like Waterfall or a Hybrid form of Agile
• Reliance on Specialists: Not all specialities represented in the agile team and therefore had to use subject matter experts outside the team for certain elements and approvals
• Complicated Architecture: Working with many different systems, tools and technology  that it was difficult to be truly cross-functional and loosely coupled

They had some success in having moved to quarterly releases from the 12 month release cycle that had been the case previously. The integration issues were being seen as one of the main barriers to moving to more frequent releases. To fully realise the benefits of moving to agile, we needed to not just manage dependencies, but to ruthlessly mitigate them. There is no silver bullet, no one-size-fits-all solution but there were some agile principles and practices we applied to mitigate and manage the integration issues. Some were quick fixes, and some required longer-term, more involved solutions as we scaled and built organisational and team capability and agile maturity. In the short term, transparency and communication is key. In the longer-term, we needed a more empowering state. We looked at how to to use agile practices to build capability of more cross-functionality,  bringing operations and development together within the team, opening up permissions, organisational design changes and evolving environment practices to move to automation and continuous integration. This required a system based approach to solving what is a large complex problem and would take effort from everyone to see successes on a wide scale.

Agile Practices we employed to Mitigate Integration Issues

Agile Principles we applied as we scaled

Transparency

We started with helping the Dependency manager (Integration and Release Manager) to visualise the dependency roadmap to increase the transparency of dependencies, potential conflicts, sequencing issues, risks and impediments, to enable their program leadership team to remove impediments that are outside the train’s control, but within the wider context of optimising for organisation-wide goals. This was the first time that the program had visualised all the delivery areas and impacted functions needed to deliver the feature from an end-to-end perspective. This transparency helped leadership understand the “hidden” risks in the delivery planning as engagement around integration was happening too late in the process and well after commitments to commitments and stakeholders had been made regarding delivery dates. This put strain on teams to meet very tight time frames. Each train and delivery area whilst dependent on each other, did no formal planning of releases as a whole and instead worked in their own component delivery silos. At scale, this transparency and collaborative approach was needed to meet organisations objectives and the program used integration and release planning prior to their “big room” program Increment ( PI) planning days to proactively start to understand and map the dependencies within and external to the program trains and teams.

Systems Thinking to take a Holistic View

ZXM guided the Integration team on a learning journey, coaching release and integration leadership to think in terms of the outcomes required for a holistic integrated release picture for the program train and the other delivery areas/trains that were required to deliver the full capability end-to-end. We had done a value stream mapping workshop with the business sowners, product management, solution design and delivery leadership and this showed that the end-to end lead time from ideation to delivery was approximately 150 days. This meant that Features on average were taking between 4-6 months before they could be released. The two main bottlenecks identified were solution design (taking a long time to design features at a high level and a tendency to do detailed design before agile teams commenced work) and integration of the work of the component teams downstream. Whilst we worked with the product management team on the solution design bottlenecks, in parallel we focuse don how to shift testing and integration left to increase time to market by reducing cycle time within a stage and overall lead time. We needed to address the bottleneck issues and map the dependencies to develop a holistic view of the program from a release planning perspective.  ZXM supported the Integration team through the process of designing and implementing an Integration workshop, including defining the outcomes required and developing the agenda for the half day workshop.

Collaboration, Engagement and Communication with Stakeholders

Ensuring a focus on engagement across the delivery trains and business lines, ZXM coaches helped leadership team develop a communications package sent out to all invitees and stakeholders concerned. Participants included staff from the program train,  impacted stakeholders such as Release Train Engineers, Product Managers, Business stakeholders, end-to-end testing elements and Change Management representatives across trains and various business lines. This enabled the workshops to move from “fact finding” to confirmation events, with exceptions worked through in greater detail right then and there, whilst major stakeholders are present. It also allowed for building greater collaboration and trust  between business, branches and trains. This ongoing engagement allowed  Integration groups to instantly highlight if a Feature or deliverable component is at risk of slipping so risks and impediments to the whole program were visualised sooner and mitigated.

Continuous Improvement

We wanted to build in continuous improvement and develop a sustainable approach that would ultimately be led by internal leadership and key personnel in agile roles. Therefore to develop and build this capability, ZXM took a staged approach to implement. ZXM Coaches facilitated the workshop over two consecutive Program Increments, involving key internal personnel in the process. This transitioned to those internal personnel leading the workshop during the third program increment, with ZXM coaches providing guidance and feedback and stepping in to assist in facilitation if required. Continuous improvement in the workshop design and approach was enabled along the way, through retrospectives, incorporating workshop participants’ feedback. By the fourth Program Increment ZXM Coaches were able to step back into a pure observer and advisor role, supporting key personnel with one-on-one coaching.

Conclusions

The solution to ruthlessly mitigating dependencies lies in empowering people and enabling flow. Over the past 12 months there has been a significant decreases in lead times and increases in the clarity of cross-branch and inter-train commitments. The improved visibility ensures dependencies and conflicts are identified earlier in the delivery cycle.  Risks, both within and across trains, are also more clearly understood resulting in informed decision making. This has also enabled better tracking of critical delivery points to ensure that potential slippages are identified earlier and follow up action can be instigated earlier than it previously could have been. Many practices that were manual have been automated (e.g shakedown and regression testing) an d there is progress being made on moving towards continuous integration. There are still challenges as the teams working on the program are always interacting and dependent on non agile hybrid teams however the move to developing and entrprise agility capability is building as majority of teams have adopted agile ways of working and are actively looking at cross skilling and building features teams across branches rather than continuing to structure themselves as component teams.