The software industry has embraced agile methods. A rising number of teams is now trying to deliver value incrementally in small iterations. This is of course a good thing. Customers get new functionality more frequently out into the market, an increasingly important factor in a competitive global economy.
The innovators and early adopters of agile methodologies were able to deliver great results using agile methodologies like Scrum and XP. But these early adopters were software craftsmen, proud developers of well-crafted software with lots of enthusiasm in addition to good communication skills. These early agile teams would probably succeed using less agile methods than Scrum and XP. Now the agile train has really started rolling, the majority has “seen the light” and even the laggards are starting to think that this might be worth looking into. Should we be happy? Or is it time to get just a little bit worried?
The challenge the software industry is facing now is that instead of getting crappy software into production once every year, we get crappy software delivered 4-12 times a year (depending on iteration lengths and a couple of other factors). But wait, this is a good thing isn’t it? We can clean up the crap, stop projects that don’t deliver value and so on. In theory yes, and I guess some organizations are able to do this. Many organizations however, do not have the required dicipline.
Teams struggle to deliver software on time with expected quality
The following is not an unlikely scenario in a typical scrum based project:
The team starts the sprint with sprint planning where they select the top prioritized stories from the product backlog. This is followed by detailed planning and estimation of tasks. The task board is set up with stories and tasks and everybody is happy. The team starts on the new sprint but after a few days it becomes clear that the team has underestimated the complexity of the stories. New tasks are identified and added to the scope. From here, the team has quite a few choices, two of them are:
- They can start to scope out stories. Or worse: parts of a story (typical if the team has not been able to split up the stories)
- Compromise on quality and allow bugs into production
I’ve seen both, and in some cases both combined. The text book states that one should never compromise on quality. However, when the organization is measured on the amount of new functionality delivered, it is easy to understand that new functionality is prioritized over bugs and technical debt. Software degrades and changing it will become increasingly difficult, causing even more bugs and technical debt. This is a negative feedback loop and breaking it can be extremely difficult.
(James Shore and Rob Myers gave a very good presentation about “Metrics in an agile world” at Agile2009, you can find it here)
Improving reliability with Kanban
Kanban can help teams build quality software that the customer wants within a predictable time frame.
An introduction to Kanban is beyond the scope of this post, but there are several good resources on the web. In the resource section at the end of this post you can find some links to online Kanban resources
Queue and WIP limits
Limits is an extremely important concept in Kanban. There are two types of limits: Queue limits and WIP limits.
Basically, queue limits will prevent you from doing too much up front planning, Corey Ladas has an excellent definition in his book “Scrumban”:
The ideal work planning process should always provide the development team with best thing to work on next, no more and no less
WIP limits on the other hand are there to limit the amount of work in progress at any given time. The morale is (again quoting Corey Ladas):
“Don’t write more specs than you can code, don’t code more than you can test and don’t test more than you can deploy.”
The Kanban board below shows the value of limits. The “pri” column holds the 3 top proritized “tasks” (bug, feature, etc..) This is enough to keep the team busy and make it easy for team members to pick new tasks. New items are prioritized into the queue before the queue is empty, usually when there is less than 2 items left. These tasks are pulled in from the “Stories in progress” station.
The development station can hold a maximum of 5 kanban cards. Setting this limit can be a challenge, but one that worked for us was to allow every developer to hold 2 cards (to compensate for blocking), but then extract one kanban card to avoid the anti-pattern of all developers holding 2 cards at all times.
System test is the next station in our process and a very important one. It can hold 3 kanban cards. Having this as an explicit part of our process have several positive effects:
- Every feature and bug is properly tested in a full system test environment
- We keep the system under relatively good control since we don’t overload the system with too many new changes before we have verified those already in the process pipeline
Here lies the beauty and the power of pull systems and limiting work in progress. If a bottleneck occurs downstream in system test, upstream station development will eventually be blocked (see figure below). This prevents us from overloading the system with new features or bug fixes before we have verified the current WIP in system test. This means that team members need to help out in resolving the bottleneck. It might not be necessary, or even possible, for all team members to help out. Instead, they can spend time improving the overall quality of the software being built (more tests, refactoring etc…)
It’s quite interesting that since the system will force us to go slower for a short period of time, this gives us the potential to go faster in the future. Limiting WIP may be hard for some teams in the beginning.
It can be especially hard to see why you should not take on more development work when the WIP limit is reached. After all, we’ve been taught that productivity is proportional with the amount of work we deliver. It takes a little while to get used to the fact that in order to go fast, it is sometimes necessary to wait.
The importance of defining testable features
An interesting learning point we made not long after starting our Kanban system was that if we wanted systemtest to be part of the process, then all our feature tasks had to be truly testable. Before, our tasks had been very focused around technology and technical details. Now we are concentrating on defining tasks that actually provide value to the customer. This also helps us focus on delivering what the customer actually wants. If the team is practicing techniques such as ATDD and/or BDD and the alike, we are improving the odds even further.
No matter how great the process or your team is: You will find bugs. So handling them in an efficient manner is important for building a quality product. Basically, bugs must treated as first class tasks just like features. If not, bugs will accumulate and the software will degrade at a much greater pace than necessary. So handling bugs sooner instead of later is almost always preferable. While there are many ways to treat bugs in a Kanban system, a detailed overview is beyond the scope of this post, but a good subject for a later post
Sceptics claim that since there are no iterations or estimation in Kanban, you don’t know when you are finished. We have already seen how WIP helps you get your features through development and test and potentially shippable. Another important concept in Kanban is lead time. This is the time it takes for a feature to get through the process once it gets in. After a while you can start to make predictions like: We can deliver user story X within 7 days. In my opinion this is far more predictable than to say: We think we can deliver stories x,y,z within 4 weeks.
Start with something small and experiment
Kanban is a lightweight process, unlike Scrum it does not prescribe much (see Henrik Kniberg Kanban vs Scrum for a great comparison of the two). This makes it pretty easy to get started. Start with applying WIP to your task board. Keep it simple and observe and reflect. When the system has stabilized, do another change and observe the result. But remember to keep the changes small. Rocking the boat too much can make it roll over.
It is my opinion that timeboxed iterations are causing more harm than good. What we see now as Agile has become mainstream, is that teams struggle to deliver within these timeboxed iterations. The consequence is less value for the customer and poor quality. I think Kanban can improve this situation, if we can:
- limit work in progress
- focus on delivering testable features and
- include testing (automatic test, system test, exploratory testing etc….) as part of the process pipeline
then we are in a better position to deliver high quality software alligned with what the customer wants within a predictable time frame.
Resources and references
- David Anderson, father of Kanban style software development, keeps a collection of Kanban resources
- Karl Scotland keeps a collection of Kanban resources here
- Henrik Kniberg has written a great article comparing Scrum and Kanban
- Corey Ladas Scrumban book is highly recommendable. You can find a link to it here
- Limited WIP society aims to be home of kanban for software development and contains a lot of great resources