Principle W5: The Batch Size Decoupling Principle in Donald Reinertsen’s excellent book The Principles of Product Development Flow states:
Use WIP ranges to decouple the batch sizes of adjacent processes
Reinertsen explains: “By using a range of acceptable WIP levels, we can use different, and economically optimal batch sizes in adjacent processes”
Here is a “real-life” example of this principle…
(SITE in this picture is what I call acceptance test in what follows)
It is also an example of an event based delivery policy. Triggering releases of software when an event occurs is one way of releasing software. Another one is by using a regular time based cadence. As mentioned above, this Kanban board shows an example of the former. We’ll soon discuss the policy associated with this, but first let’s take a look at the problem we are trying to solve.
Big Releases Decrease Stability Through Increased Complexity
Our development process runs quite smoothly up to the acceptance test point. Prioritization is triggered when number of tickets drops below a certain amount, in our case this number is 3. When a new feature, bug fix or improvement is finished, it is checked in. This triggers a build on the Hudson continuous integration server which again triggers a downstream build responsible for deploying the latest build to the system test server.
Ok, so far so good. Moving releases to the acceptance test environment is a far more rigid process though. The actual build and deploy of artifacts is done by a separate unit in the organization. The deploy needs to be ordere, and according to the SLA the unit has a couple of days to complete the request. In times of high demand for these kind of services, this unit serves multiple lines, there is a reasonable risk of delay. This represents an obstacle for progressing into the acceptance test environment. As a consequence, work items tend to pile up in the queue between these two adjacent processes. The consequence is increased queue sizes in the system and increased cycle time.
Adding to the problem is the fact that the business side is under the impression that doing frequent releases has a negative impact on the production environment. There seems to be a lack of trust between the business side and the development team, probably caused by problematic releases in the past. Consequently, stakeholders have wanted to minimize the number of such incidents by keeping the frequency of releases down. The result of this has been big complex releases which in fact have increased the probability of failure.
Event Triggered Releases
Of course those of us who has worked in environments with frequent releases knows that is exactly the opposite: Frequent releases leads to less complexity and thus increase the chance of success when releasing the software. In one of my previous posts, Improving Software Quality with Kanban and XP Practices I have described how and why this is a better solution.
So how can we get this message through to the business side? We could of course explain them the theory behind why frequent releases works. And we might even be able to convince them. But a far more effective way, in my opinion, is simply to show them. But how? In an environment where quarterly releases is the norm (best case), asking for monthly releases seemed to be far fetched. So we decided to go for an event based solution:
There is a certain transaction cost associated with deployment to the acceptance test environment. Too small batches means that the transaction cost will be high. Too large batches results in increased complexity, leading to more defects in the environment. So to keep the risk profile of a release at an acceptable level and at the same time respect that there is a transaction cost associated with it, we have introduced a policy that a release package must have minimum 5 items and have maximum 10 items.
This is quite a different approach than the more standard time driven cadence. It gives us the ability to deliver features/bug fixes whenever there is enough ready to be delivered. But as David Anderson pointed out at the Kanban Training Class last week, this will increase the coordination cost of doing releases. When a release package is assembled the team needs to communicate the release both with upstream and downstream processes. It requires more of the organization and is therefore probably a better fit for mature organizations.
We only just started using this event based triggering of releases, it will definitely be interesting to see the result over the next couple of months. We hope that it will help us achieve our goal of keeping releases small. Hopefully it will help us convince stakeholders that moving towards more frequent releases is a good thing and that it over time will improve the stability in the production environment.