My undergraduate education had a strong foundation in systems thinking – it is one tool that is particularly well aligned with the work of Human Ecologists, for helping us understand complex interactions in social-ecological systems and mechanisms for change. I came, naively, to evaluation thinking that I would be able to apply these big systems thinking approaches (e.g. soft systems methodology, critical systems heuristics) on a large scale in evaluations. That is, that I would be writing them into methodologies, that clients would be lapping them up, excited to see the possibilities for reflective practice. In reality, this reflects my initially instrumental understanding of systems thinking!

It’s also why I was so excited when I discovered that there is a body of literature around systems concepts and thinking in evaluation – bringing two interest areas of mine together. Bob Williams and Iraj Imam’s anthology of systems thinking in evaluation is a great place to start for this, along with a lot of the work by Gerald Midgley in this area. These offer much for consideration in evaluation. However, whilst reading them, I did find there was something missing. I knew from my everyday evaluation practice – because of client expectations, resource constraints, expectations around time –  that implementing an evaluation based on systems thinking might be a challenge if I were to try it some day. I was left wondering how I could actually do this in my day to day work. I wanted something more.

I recently came across this useful model from the Waters Foundation, which describes 13 ‘habits’ of systems thinkers.

Habits of a systems thinker

 

 

 

It got me thinking about the role of systems thinking in evaluation. I like that it makes systems thinking personal, rather than some abstract methodology to be implemented. It struck a chord, because I realised where I’d been going wrong – I’d been trying to find models or frameworks with a systems thinking ‘flavour’ to apply to evaluations. Instead, like all good things, starting with the self was where systems thinking was going to come in handy! That is, I could be a systems thinker, but not necessarily do systems evaluation per se. The habits listed above are tangible and easily practice and could be closely aligned with many of the commonly understood evaluation competencies.

Whilst all could have a bearing on my evaluation practice, those that stand out to me are:

Finds where unintended consequences emerge

Oops! A double rainbow!

My excitement at seeing the first rainbow drew my attention to the second!

I consider this an important aspect of good evaluation – to not just look for the expected or intended outcomes of an intervention, but also understand unexpected outcomes. Thinking in this way can also goes some way in developing alternative explanations for causal inference and considering the wider relationships between the outcomes and other ’causes’.

Surfaces and tests assumptions

Will the road really take you where you intend? Questioning where your line of thinking is taking you takes effort!

Will the road really take you where you intend? Questioning where your line of thinking is taking you takes effort!

So much about evaluation, to me, seems to be about making the implicit, explicit. So, habitually working to draw out assumptions and test them, with others, is core to being a useful evaluator. This can be done in conversation, in daily life – it’s not just useful at the scale of program evaluation.

Seeks to understand the bigger picture

Seeing the whole forest, as well as the trees

Seeing the whole forest, as well as the trees

I also feel that in daily evaluation practice, I can get bogged down in the details and busy-ness of what I need to get done. Forgetting the bigger picture is dangerous as you can miss important information! So, making a daily habit of seeking the bigger picture and questioning the place of small things in their context might help keep things in perspective.

So, I am going to spend some time experimenting with ways of practicing these habits in my work and life in a more explicit way. They are habits that I feel intuitively drawn to, but actually drawing them out, articulating and critiquing them is the next step for making a more reflective practice!

Do any of the habits stand out to you? How could you use these in your work or life?

Update: Thanks to some great conversations after posting this blog, I’d like to add this description of thinking systemically from Bob Williams which goes further in describing some ways of using systems concepts. I see this as a sort of ‘level up’ from the schematic I was discussing in this post, in terms of going deeper into the background on systems thinking (and also resolving some common misconceptions). It’s also important to note that the Waters Foundation ‘habits’ cover relationships, but not boundaries, an equally important systems concept. It’s got me thinking about what ‘boundary’ habits might look like, so perhaps there’s a future blog post in that! Thanks Bob & Better Evaluation!

 

 

7 thoughts on “Wearing your systems thinking hat: habits for better evaluation

  1. Thanks! Certainly agree that it is great to find unexpected outcomes that sit outside a Theory of Change. The Most Significant Change technique (http://www.mande.co.uk/docs/MSCGuide.pdf) is a useful way to explore those community voices you mention! Another approach I’ve found interesting – the Overseas Development Institute have developed an approach called ‘episode studies’ (http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/publications-opinion-files/7072.pdf) which is a sort of trace back approach from instances of change, to see if they have links with your intervention. It’s a kind of reverse way of tracing both unexpected and expected outcomes at a policy/practice scale!

    • Thanks, I love ODI… great resource, will look into episode studies particularly. I try to use Empowerment Evaluation principles plus was looking to combine Outcome Harvesting and Contribution Analysis (great definitions at http://betterevaluation.org/ as am sure) to look backwards and make some sense of contribution/ attribution in ex-post evals. Cheers!

  2. Refreshing to see and hear. I have been connecting these in my own work for several years– having attended the Systems Thinking in the Schools workshops and Camp Snowball twice. The team from the Waters Foundation have been great. Do you participate in the Systems TIG of the American Evaluation Association?
    Thanks for writing and sharing about these topics!

    • Thanks for your encouraging comment Allison. What ways have you connected them in your work? I’d be fascinated to hear!! And, I will take a look at the Systems TIG and options for participating. Cheers, Meg.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s