Learning About Lean and Agile Software Development

13. May 2011

This year I have started giving a presentation for No Fluff Just Stuff conferences that introduces many of the most critical ideas from lean software development, and provides some practical ways to start implementing them in whatever project context you find yourself: agile, traditional, or otherwise. Here is the presentation abstract:

Successful software development is about building the right product at the right time for your customers. This means focusing attention on the right places in the portfolio of projects and products that your company provides, and optimizing the entire value stream from "concept to cash" for your customers and the development teams.

 

Agility is more than just adopting Scrum or some other agile process framework; it involves adopting a new set of Lean-Agile values, principles and practices through the entire software development lifecycle and beyond in order to provide value to customers earlier and more often.

 

Lean-Agile software development consists of frequent feedback loops, intense team collaboration, continuous improvement, business and customer involvement, baking quality in and consistent delivery of valuable software. Learn how these Lean principles and practices transform software development and the radical difference it can make in your development work and wider organization.

Lean software development is a subset of Lean Product Development, not Lean Manufacturing. It is foolish to blindly apply Lean Manufacturing practices to software development. The underlying principles of value, flow, pull and waste remain the same, but the way these principles are applied to software development will look fundamentally different.

As Paul Hodgetts points out in his excellent Lean is More white paper:

Treating software development as Lean Manufacturing leads us down a path of optimizing the mechanics of developing software, which can yield limited benefits but fails to address the much larger software product development issues.

I would put it even more strongly than this. Most of what software developers do is invisible, particularly when it comes to the thinking involved with design.

It's just not about the mechanics of the process, any more than writing a song is about the mechanics of playing the instrument or enjoying a meal is about the nutritional content of the food. Such reductionism is a false hope for improvement. For example, seeking to create a more efficient process by reducing or eliminating design-related activities in software development as waste (such as design meetings, writing design documents, refactoring etc) will inevitably lead to a product with little conceptual integrity, and fast-track it towards becoming a Big Ball of Mud. Asking developers to track hours as a measure of productivity is the fast-track to disfunction and decreasing productivity. Let's not reintroduce Taylorism to software development through the back-door of Lean.

Agile methods were developed by assembling the best practices from successful projects. While the combinations of best practices found in agile methods exhibit the core values and principles of a lean approach, it can be difficult to understand the reasons why agile processes work just from examining their practices...Mapping agile practices to lean concepts such as value, flow, pull and waste can help explain why agile works, and offer new insights to guide process improvements. (Hodgetts, p. 3)

To see this more clearly we can look through at Kanban and Scrum the lens of lean and agile principles. Kniberg and Skarin, in Kanban and Scrum: Making the Most of Both, point out that Scrum and Kanban are both aligned with the values and principles of both Lean and agile. For example:

  • Scrum and Kanban are both pull scheduling systems, which corresponds to the JIT (Just in Time) inventory management principles of Lean. This means that the team chooses when and how much work to commit to, they "pull" work when they are ready, rather than having it "pushed" in from the outside. Just like a printer pulls in the next page only when it is ready to print on it (although there is a small and limited batch of paper that it can pull from).
  • Scrum and Kanban are based on continuous and empirical process optimization, which corresponds to the Kaizen principles of Lean.
  • Scrum and Kanban emphasize responding to change over following a plan (although Kanban allows faster response than Scrum), one of the four values of the agile manifesto.

While the batch sizes in Scrum are much larger (batching work into timeboxed iterations) and thus not as lean as Kanban, producing shippable code every 2 weeks is certainly much leaner than a more traditional process which might integrate and release something 2-4 times per year. The shorter you make the iteration, the more you are approaching Kanban.

As a DDD practitioner, one area where I think the careful application of Lean thinking holds a great deal of promise is in strategic design:

Agile and lean are not in conflict with each other as they may seem at first glance, nor are they incompatible approaches to process improvement... Lean Product Development offers several strategies and practices that can supplement agile methods, particularly in areas where agile methods have been criticized as lacking...the strategy of set-based design offers a balanced alternative to heavyweight up-front design approaches on one hand, and reactive, emergent approaches on the other." (Hodgetts, p. 3)

Implementing Lean Software Development

Instead of teams feeling that they constantly have to adopt expedient design approaches to meet Sprint deadlines, Lean thinking combined with strategic DDD frees them to focus their design work where it matters most for the competitive advantage of their business. It encourages them to reduce the amount of code they write themselves, cultivate domain knowledge, and explore models collaboratively with the business to maximize the potential to develop innovative custom software solutions that leap ahead of the competition in the marketplace.

To learn more about Lean Software Development, I highly recommend Implementing Software Development by Mary and Tom Poppendieck. This book is the definitive work on the subject at the moment.

How do we get our organization, especially our stakeholders, on board with strategic design?

7. May 2011

Great question. Most stakeholders would be delighted that you are seeking to distill down your core domain. Not that they would express it in DDD terms, but rather that want to see that development effort is being apportioned appropriately according to the business strategy.

One approach would be to start by writing out the major business capabilities you are trying to achieve with each project. Put each one on a sticky note, and assign it to a quadrant in the Purpose Alignment Model.

Try to mark each capability according to the approach you are taking - are you currently treating each as core domain, supporting subdomain, or generic subdomain? Look for inconsistencies. For example, do you have business capabilities in the Maintenance quadrant? I would recommend killing all custom software development in this quadrant. Outsource those capabilities as soon as possible. Don't waste time on custom development. Don't waste time trying to support those capabilities at all.

Do you have parity capabilities that you are trying to implement using a core domain approach? Over-investment! Look for ways to use open source or 3rd party tools for generic subdomains. Hire contractors, and free up your in-house developers for supporting subdomains.

Do you have differentiating business capabilities that are not mission critical for your business that you are currently doing in-house? Find a business partner that excels at these. Ask for recommendations - who in the industry is best at this? Approach them about collaborating on a solution.

Like many development divisions/teams, you may be drowning in parity work. Trying to be too creative in your software development efforts for supporting subdomains and doing custom development for generic subdomains will leave little time for creativity in the core domain. Do everything you can to eliminate waste in your project portfolio by aligning effort and innovation to purpose. If you could reduce the amount of parity work on supporting and generic subdomains even a small amount, this would start to free up time, talent and energy to focus on your core domain.

 

Does the Core Domain Change Over Time?

7. May 2011

Does the Core Domain change over time?

Yes, it does. As the marketplace changes and the business adapts to those changes, it must adopt new strategies to gain market share and enter new markets. Strategy changes will typically lead to Core Domain changes, as the areas of the software that support business activities that generate differentiation in the marketplace also change.