Agile, Kanban

Kanban – the next step in the agile evolution?

The software industry has embraced agile methods. A rising number of teams is now trying to deliver value incrementally in small iterations. This is of course a good thing. Customers get new functionality more frequently out into the market, an increasingly important factor in a competitive global economy.

The innovators and early adopters of agile methodologies were able to deliver great results using agile methodologies like Scrum and XP. But these early adopters were software craftsmen, proud developers of well-crafted software with lots of enthusiasm in addition to good communication skills. These early agile teams would probably succeed using less agile methods than Scrum and XP. Now the agile train has really started rolling, the majority has “seen the light” and even the laggards are starting to think that this might be worth looking into. Should we be happy? Or is it time to get just a little bit worried?

The challenge the software industry is facing now is that instead of getting crappy software into production once every year, we get crappy software delivered 4-12 times a year (depending on iteration lengths and a couple of other factors). But wait, this is a good thing isn’t it? We can clean up the crap, stop projects that don’t deliver value and so on. In theory yes, and I guess some organizations are able to do this. Many organizations however, do not have the required dicipline.

Teams struggle to deliver software on time with expected quality

The following is not an unlikely scenario in a typical scrum based project:

The team starts the sprint with sprint planning where they select the top prioritized stories from the product backlog. This is followed by detailed planning and estimation of tasks. The task board is set up with stories and tasks and everybody is happy. The team starts on the new sprint but after a few days it becomes clear that the team has underestimated the complexity of the stories. New tasks are identified and added to the scope. From here, the team has quite a few choices, two of them are:

  1. They can start to scope out stories. Or worse: parts of a story (typical if the team has not been able to split up the stories)
  2. Compromise on quality and allow bugs into production

I’ve seen both, and in some cases both combined. The text book states that one should never compromise on quality. However, when the organization is measured on the amount of new functionality delivered, it is easy to understand that new functionality is prioritized over bugs and technical debt. Software degrades and changing it will become increasingly difficult, causing even more bugs and technical debt. This is a negative feedback loop and breaking it can be extremely difficult.

(James Shore and Rob Myers gave a very good presentation about “Metrics in an agile world” at Agile2009, you can find it here)

Improving reliability with Kanban

Kanban can help teams build quality software that the customer wants within a predictable time frame.

An introduction to Kanban is beyond the scope of this post, but there are several good resources on the web. In the resource section at the end of this post you can find some links to online Kanban resources

Queue and WIP limits

Limits is an extremely important concept in Kanban. There are two types of limits: Queue limits and WIP limits.

Basically, queue limits will prevent you from doing too much up front planning, Corey Ladas has an excellent definition in his book “Scrumban”:

The ideal work planning process should always provide the development team with best thing to work on next, no more and no less

WIP limits on the other hand are there to limit the amount of work in progress at any given time. The morale is (again quoting Corey Ladas):

“Don’t write more specs than you can code, don’t code more than you can test and don’t test more than you can deploy.”

The Kanban board below shows the value of limits. The “pri” column holds the 3 top proritized “tasks” (bug, feature, etc..) This is enough to keep the team busy and make it easy for team members to pick new tasks. New items are prioritized into the queue before the queue is empty, usually when there is less than 2 items left. These tasks are pulled in from the “Stories in progress” station.

Kanban board

The development station can hold a maximum of 5 kanban cards. Setting this limit can be a challenge, but one that worked for us was to allow every developer to hold 2 cards (to compensate for blocking), but then extract one kanban card to avoid the anti-pattern of all developers holding 2 cards at all times.

System test is the next station in our process and a very important one. It can hold 3 kanban cards. Having this as an explicit part of our process have several positive effects:

  • Every feature and bug is properly tested in a full system test environment
  • We keep the system under relatively good control since we don’t overload the system with too many new changes before we have verified those already in the process pipeline

Here lies the beauty and the power of pull systems and limiting work in progress. If a bottleneck occurs downstream in system test, upstream station development will eventually be blocked (see figure below). This prevents us from overloading the system with new features or bug fixes before we have verified the current WIP in system test. This means that team members need to help out in resolving the bottleneck. It might not be necessary, or even possible, for all team members to help out. Instead, they can spend time improving the overall quality of the software being built (more tests, refactoring etc…)

Kanban_board_blocked

It’s quite interesting that since the system will force us to go slower for a short period of time, this gives us the potential to go faster in the future. Limiting WIP may be hard for some teams in the beginning.
It can be especially hard to see why you should not take on more development work when the WIP limit is reached. After all, we’ve been taught that productivity is proportional with the amount of work we deliver. It takes a little while to get used to the fact that in order to go fast, it is sometimes necessary to wait.

The importance of defining testable features

An interesting learning point we made not long after starting our Kanban system was that if we wanted systemtest to be part of the process, then all our feature tasks had to be truly testable. Before, our tasks had been very focused around technology and technical details. Now we are concentrating on defining tasks that actually provide value to the customer. This also helps us focus on delivering what the customer actually wants. If the team is practicing techniques such as ATDD and/or BDD and the alike, we are improving the odds even further.

Handling rework

No matter how great the process or your team is: You will find bugs. So handling them in an efficient manner is important for building a quality product. Basically, bugs must treated as first class tasks just like features. If not, bugs will accumulate and the software will degrade at a much greater pace than necessary. So handling bugs sooner instead of later is almost always preferable. While there are many ways to treat bugs in a Kanban system, a detailed overview is beyond the scope of this post, but a good subject for a later post

Lead time

Sceptics claim that since there are no iterations or estimation in Kanban, you don’t know when you are finished. We have already seen how WIP helps you get your features through development and test and potentially shippable. Another important concept in Kanban is lead time. This is the time it takes for a feature to get through the process once it gets in. After a while you can start to make predictions like: We can deliver user story X within 7 days. In my opinion this is far more predictable than to say: We think we can deliver stories x,y,z within 4 weeks.

Start with something small and experiment

Kanban is a lightweight process, unlike Scrum it does not prescribe much (see Henrik Kniberg Kanban vs Scrum for a great comparison of the two). This makes it pretty easy to get started. Start with applying WIP to your task board. Keep it simple and observe and reflect. When the system has stabilized, do another change and observe the result. But remember to keep the changes small. Rocking the boat too much can make it roll over.

Conclusion

It is my opinion that timeboxed iterations are causing more harm than good. What we see now as Agile has become mainstream, is that teams struggle to deliver within these timeboxed iterations. The consequence is less value for the customer and poor quality. I think Kanban can improve this situation, if we can:

  • limit work in progress
  • focus on delivering testable features and
  • include testing (automatic test, system test, exploratory testing etc….) as part of the process pipeline

then we are in a better position to deliver high quality software alligned with what the customer wants within a predictable time frame.

Resources and references

  • David Anderson, father of Kanban style software development, keeps a collection of Kanban resources
  • Karl Scotland keeps a collection of Kanban resources here
  • Henrik Kniberg has written a great article comparing Scrum and Kanban
  • Corey Ladas Scrumban book is highly recommendable. You can find a link to it here
  • Limited WIP society aims to be home of kanban for software development and contains a lot of great resources
  • Slidehunter has some Free Kanban Powerpoint Templates you can use to create mock-ups of boards
Advertisements

22 thoughts on “Kanban – the next step in the agile evolution?

    1. Thanks, Balsamiq mockups makes the job of drawing mocups easy.

      I’ve added Limited WIP society to the list of resources. Looks like it has a lot of great resources.

  1. Ketil,

    Great overview post, and thank you for sharing this. I do have a small point I’d like to raise, although it’s less with you than with the kanban community in general. In your conclusion you state “It is my opinion that timeboxed iterations are causing more harm than good.”

    I feel that we are not doing ourselves a favor when we make statements like this. I recognize your intent, which I believe is to say “It is my opinion that using a pull based approach provides more value than using iteration based approaches while still encouraging the small batch sizes, focus on achieving a done state, and the other improvements over waterfall that iterations allow.”

    Within a community, I believe most people think this way and the “more harm thatn good” is a shorthand for the relative nature of the intent. But when speaking to a broader audience, I feel it is important for our communities to be more explicit in recognizing the value that agile approaches in general have provided.

    I encourage people exploring new techniques and approaches to be very explicit when stating which practices the new one should replace, and talk about how the new practice accomplishes the same goals using different mechanisms. (Or, put differently, satisfies the same priciples using different pratices that still adhere to the overall values)

    Again, thank you very much for sharing this overview. I do greatly appreciate seeing people share the many perspectives on kanban in well-written posts like this one.

    1. Eric,

      Thank you for your comment. Glad you liked the post, and I really appreciate the feedback.

      You’re pretty much right about my intent, well put. My goal was to make a concise to-the-point conclusion. I see you’re point about making the intent explicit when speaking to a broader audience.

  2. Great post Ketil. I’ve found with the teams that I work on that they usually have a driver (business) that pushes features over quality, and that as you state, quality degrades and technical debt mounts up. I find it amazing that managers continue to focus on features rather than quality, but I can understand why, though I don’t agree with it. Regardless, I like the approach you mention here of small batch sizes, limiting WIP, allowing teams to wait in order to speed up, and “forcing” a feature through the process, ensuring too that the testing gets put in place. Excellent stuff and something I’m going to try with my next team.

    1. Thanks Robert, pleased to hear that you found the post useful.

      Like you say: One can understand that business wants new features out instead of fixing bugs or improve quality. The problem is that after a while it’s going to be harder and harder to add new features to the system. Maybe this won’t hit you during the project, especially not if it is a small project. But sooner or later this is going to happen. Limiting wip is, in my opinion, one really good way of increasing the chance of delivering software with high quality.

      Would be very interesting to hear about the results if you decide to try it out on your next team.

  3. Sounds great and I’ll love to try it out in my next project!

    I have one question I couldn’t quite figure out about Kanban. I don’t get if you guys are using User Stories as cards, or if you use Tasks. Do you keep tasks together or just let them move around “randomly” on the board? How can you track which tasks belongs to what story?

    1. Thank you for your comment,

      Those are really good questions. Unfortunately, I don’t get into details about these things in the post. This is partly due to the fact that I had to draw the line somewhere. Also, I think this is one of the more challenging aspects of applying Kanban and one that we have struggled quite a bit with.

      Now, the team I’m working on had been using Scrum for a while but never used Kanban. When we decided to go for Kanban we agreed that we needed to do some small changes first, specifically we decided to start by limiting WIP and add system test as part of the development process. As mentioned in the post, by doing this we realized fairly quickly that we needed to make the tasks truly testable.

      “I don’t get if you guys are using User Stories as cards, or if you use Tasks”
      So, are we moving tasks or stories around? Well, I guess it’s something in between, some of them are perhaps small user stories (but we are not using user story syntax for these). I have now started to refer to them as features cause I feel that this reminds us that they need to be testable. This means that the cards with text “User story” in the illustrations above are more like Epics or Use cases, so I can see that this is a bit misleading

      “Do you keep tasks together or just let them move around “randomly” on the board?”
      This is what the “swimlanes” are for. Each feature (or bug for that matter) belongs to the swimlane given to the story (or epic/use case) as long as it is being developed. We currently don’t use swimlanes when in systemtest, I can see that we could have… Just had a short discussion with the team and found out that there is really not a reason for not extending the “swimlane” into system test so I think we will apply this change.

      “How can you track which tasks belongs to what story?”
      We have now started to tag each card with a number referencing the epic/use case. This takes care of tracking of the features. We don’t

      We are continously reflecting over our process to see if we can improve and I think this is important. Start with some small changes and then reflect and learn.

      1. Ah, thanks a lot for the answer!

        Swimlanes sounds like a good idea. Haven’t seen that before though. It looks like it would be possible to dedicate each user story (and its tasks) to one swimlane just like in your picture.

        But how do you handle stuff like acceptance testing individual tasks? I tend to think that user/acceptance testing should be done on the “completed” story (ie customer value/MMF) and not individual development tasks. Wouldn’t that mean that you have to queue up and block the pipeline until all tasks in the story is done, and _then_ move the whole story into the acceptance testing station?

        I guess the same reasoning applies to the systems testing station as well. But I guess, in some cases, it is possible to test individual development tasks separately before the story is done. (I’m of course assuming we are using TDD during the dev station)

        By the way, my team usually create stories that takes 0.5-3 days to complete. Each task is at most estimated at 4 hours or something. Maybe that is to small items for using in Kanban? But on the other hand, larger items tends to end up being much harder (or easier!) to complete.

      2. I don’t think Kanban prescribes anything with regard to task size. What I’ve read from others though is that it is an advantage to try and keep items approx the same size in order to get predictable lead times.

        You’re right about the acceptance test. In our case this is indeed decoupled from the rest of the process. So what we say is that when we’re ready for AT then we are “done” with the feature. In an ideal world we would like to push the feature all the way into production, but this has proved difficult to accomplish. You could then argue that the kanban board should not show the process steps after “Ready for at” , but by showing these process steps we accomplish two things:

        1) We get a good overview of which features are ready for deployment and
        2) We visualize that there in fact is a queue where features are “stocked”, waiting to be deployed into AT. In a learning organization this can result in a process improvement in the future.

        When it comes to system test, I think that you can test individual features (at least if you focus on making them testable). For us, this was a big improvement and by treating system test as part of the process pipeline we have fewer bugs when moving into AT then in previous projects.

  4. An excellent article on Kanban (this is actually the first time I hear about this concept).

    Ketil,

    I would love to republish this article on PM Hut, as I think it would give Project Managers an excellent insight on the subject. Please either email me or contact me through the “Contact Us” form on the PM Hut site in case you’re OK with this. Thank!

  5. Ketil,

    Nice write up. Like David I really like your balsamiq whiteboard mockup.
    Do you have the bmml files available for download?

    Greetz….
    Maarten

    1. Thanks Maarten.

      Unfortunately I have no bmml files available, if I had I would be happy to share. Since writing this post I’ve tried to use Balsamiq Mockups for drawing kanban boards on other occasions, but found that it’s kind of tedious to use. In a post published a few months ago I used Visio instead.

      Thank you for your interest.

      Cheers,
      Ketil

    1. Hi,

      Sorry for the late reply. I will add your link to the list of resources as this could provide value to people reading the post. Thanks for reading 🙂

      Cheers,
      Ketil

  6. I believe that avoiding ready-made foods is the first step to lose weight. They can taste very good, but processed foods currently have very little nutritional value, making you feed on more only to have enough electricity to get over the day. When you are constantly taking in these foods, transitioning to grain and other complex carbohydrates will let you have more vitality while taking in less. Thanks alot : ) for your blog post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s