Complexity: Justify, Explain or Apply?

Any idea, theory or science can be used to:

  1. Justify what you’re doing now, or would like to start doing
  2. Explain previous events, why some things fail and others succeeded
  3. Applied to create or synthesise new approaches to problems you have today

All are valid, and do not exist in isolation; rather they exist as overlapping zones in a continuum , and you can move between them at will.

Complexity Science’s oldest and most loyal cheerleaders are probably the Scrum Community; Complexity Science gets a mention in Ken’s landmark book “Agile Software Development with Scrum” – and the Stacey Matrix appears in the majority of Scrum courses today.  Scrum is touted as “a Framework for Product Development in the Complex Space” and the Stacy Matrix serves well to explain what the Complex Space is and why you need a new approach to dealing with it.

So this is useful, but ultimately it is simply using Complexity Science as a means to justify changing the approach used to manage Software Product Development (or anything else that Scrum is being applied to these days)

Nobody sane makes a change without some kind of Justification,  so this has been and still is a valid and useful technique.

Problems can occur however thanks to our friend Confirmation Bias.  If you like Scrum then you’ll want to use it all the time, and because of this you’ll be biased  towards seeing every problem as “Complex” so that you can use your favourite tool on it.  You have effectively created a tiny mental prison for yourself.

“So what!?”, you might ask…

in which case I would reply: Scrum has been around for 15 years or so, and we’ve been attaching the “Agile” label to things for the last 10, and the results have been rather varied.  Panacea it most certainly isn’t.

But why?

This sort of question has led myself (and others) to delve deeper into the techniques we’ve been using, the underlying theories (if any) they were based on, and to seek out the latest research in a quest to truly understand what is going on, and what expectations we should have.

This search led me through a personal journey beyond Stacey and into the wonderful (but hard to digest) world of the late Max Boisot and his I-Space model and then ultimately onto the current cool kid on the Complexity Block Dave Snowden’s Cynefin.  I then delved into works of Baumeister, Kahneman, Tversky et al to better understand the limits of our cognition, how we make decisions and effective mechanisms to overcome procrastination.

I used this research to understand what the Scrum practices were trying to achieve, and what effects they actually had. I then extended this knowledge to create a model to explain why Scrum (as interpreted solely by its practices) will thrive in certain situations and self immolate in others.   I presented these findings in a workshop I held at Agile 2011 called “Scrum as a Thinking Toolkit”

This is just one example of moving from Justification (where we know the right answer and seek information only to confirm our existing beliefs and desired actions) – towards explanation – the creation of models & theories which help us understand why things have historically gone the way they did and thus hopefully provides useful insights into dealing with the future.

Once you have begun to understand, you can then begin to Apply.  Once you know the “Why” you can begin to create your own “How” – and even better, you can  take advantage of the considerable advances that have been made in the area of human cognition and social complexity to enhance your work.  As such, among other things, I began to apply “SenseMaking” to the management of Backlogs and Ritual Dissent to Release Planning.

And you know what I found?

Hey this stuff works…

This is the philosophy that is at the root of CALM (Complexity Agile Lean Mashups) – that of “theory informed practice” – synthesise new practices based on well researched principles and then validate these practices in the field.  Loop back and continue.

It’s this level of understanding that will help you understand why in some projects removing the time boxes made things improve, whilst in others it led to anarchy and disaster.

So I can imagine what some of you are currently thinking:

“Hey! Complexity Science was originally applied to create Scrum in the first place!  It’s built in! I don’t need this!”

And that’s exactly my point.  Regardless of where we mark the Scrum epoch (somewhere between 1987 and 1994 I believe is the currently accepted range) – Science has moved on.  Many of the models used to originally describe (and thus by association define) Scrum are now dated at best and debunked at worst.  We know so much more now.  The heartening thing is that using our current best available knowledge the “old girl” actually stands up very well for her stated mission.

But there is still more to have – much of software today exists outside of the Complex realm, there are other containers than time boxes, there is knowledge out there that scales far beyond the methods we’re using today; and perhaps most importantly there are strategies and techniques which can help us address the wider organisational issues beyond just those we encounter in our teams and on our projects.

CALM is about making a move from using science to justify what we already want to do, towards a mindful application of theory; a cycle of evolution and synthesise where practice and theory co-evolve to help us realise our true potential.

The first CALM event, CALM Alpha will be held in the UK on the 16th and 17th of Feb 2012.

2 thoughts on “Complexity: Justify, Explain or Apply?

  1. Harold van Garderen

    Hi, Nice blog, small remark on “Complexity Science’s oldest and most loyal cheerleaders are probably the Scrum Community”. Ken’s book is A.D. 2001.

    I’m quite sure, chemists were a bit earlier. Have a look f.e. at Ilya Prigogine in his 1981 book “From Being to Becoming: Time and Complexity in the Physical Sciences”, the 1985 “Order Out of Chaos “followed by “Exploring Complexity: An Introduction ” in 1989.

    The latter is an introduction, indicating the application of complexity to chemistry and physics was quite mature in 1989. This is further testified by Ilya’s Nobel price in 1977 and the fact that humble me did his PhD between 1991 and 1995 on the boundary between chemistry, physics and informatics 🙂

    Reply
  2. Simon Bennett

    Hi Harold,

    Good catch, I probably should have said “In the Software Development & Project Management Communities Complexity Science’s oldest and most loyal cheerleaders are probably the Scrum Community;”

    Kind Regards,
    Simon

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *