Continuous Delivery unaccompanied by organisational change will not reduce cycle time
Our Continuous Delivery value proposition describes a goal of reducing cycle time – the average time for a software release to propagate through to Production – in order to improve our time-to-market, saving time and money that can be invested back into product development and growing revenues. However, it is important to bear in mind that like any cross-organisation transformational programme Continuous Delivery is susceptible to Conway’s Law:
Any organisation that designs a system (defined broadly) will produce a design whose structure is a copy of the organisation’s communication structure
This extraordinary sociological observation predicts that multiple teams working on the same problem will produce disparate solutions, and that the structure of an organisation must be adaptable if product development is to remain sustainable. As a Continuous Delivery pipeline will likely traverse multiple organisational units (particularly in silo-based organisations), these are pertinent warnings that were addressed by Dave Farley and Jez Humble in the principles of Continuous Delivery:
- Repeatable Reliable Process
- Automate Almost Everything
- Keep Everything In Version Control
- Bring The Pain Forward
- Build Quality In
- Done Means Released
- Everybody Is Responsible
- Continuous Improvement
The majority of these principles are clearly focussed upon culture and behaviours, yet some Continuous Delivery implementations are entirely based upon Reliable Repeatable Process and Automate Almost Everything at the expense of more challenging principles such as Everybody Is Responsible.
For example, in our siloed organisation we are asked to improve the cycle time of an application from 28 days to 14 days, with the existing deployment and migration mechanisms manual processes that each take 20 minutes to perform. We introduce a Continuous Delivery pipeline in which we Automate Almost Everything, we Keep Everything In Version Control, and we establish our Repeatable Reliable Process. However, despite deployment and migration now taking only 5 minutes each, our cycle time is unaffected! How is this possible?
To explain this disheartening situation, we need to use Lean Thinking and examine the value stream of our application. While our new release mechanism has reduced the machine time of each pipeline stage (i.e. time releasing an artifact), the process lead time (i.e. time required to release and sign off a artifact) is largely unaffected. This is because process lead time includes wait time, and in a siloed organisation there are likely to be significant handoff periods both during and between pipeline stages which are “fraught with opportunities for waste“. If the deployment and migration mechanisms have each been reduced to 5 minutes but a 3 hour handoff from server administrator to database administrator remains, our Repeatable Reliable Process will never affect our cycle time.
To accomplish organisational change alongside Continuous Delivery, the most effective method of breaking down silo barriers is to visualise your value stream and act upon waste. Donella Meadows recommended that to effect organisational change you must “arrange the structures and conditions to reduce the probability of destructive behaviours and to encourage the possibility of beneficial ones“, and a pipeline containing a Repeatable Reliable Process is an excellent starting point – but it is not the end. Visualise your pipeline, educate people on the unseen inefficiencies caused by your organisational structure, and encourage an Everybody Is Responsible mentality.
Any devops team in the world can set up a CD pipeline in about 5 minutes. In fact, I saw one team do that on Saturday. The real skill to pulling CD off is, as you said, changing the organisation so that it supports this and helping the dev team – which you didn’t say – write better tests.
Speaking of lean, BTW, the CD pipeline can be a Target Condition that drives improvements. Once you attempt it the real organisational issues will become obvious. (My latest attempts to do CD revealed time-management problems in the MT and a weak corporate vision. So, I have to fix those before coming back down to the dev team.)
Thanks for that. Yes, a CD pipeline can be built far more quickly than an organisation can be changed… particularly if it is a single application pipeline.
You’re right to stress the importance of safeguarding a pipeline with better quality acceptance tests. For brevity I wanted to focus more upon Conway’s Law and Lean Thinking in this instance, but it is indeed a key aspect. Because the organisation I work for already has development teams writing outstanding acceptance tests with in-team BAs, it’s easy to take that for granted.
Rather than view the pipeline itself as a Target Condition, I tend to set short- and long-term goals with Operations (e.g. “manage this new platform of applications”, “reduce error rate of deployments by X”), and then use the pipeline to implement those goals.