On a recent 2-day organisation design programme we started to discuss the outcomes of organisation design work – how we identify them and how we know that these are creating the intended benefits and business value.
We agreed with one writer’s definitions and statements that:
‘Outcomes, at the most general level, are changes in individuals, organizations, communities, or governments, depending on the goal and reach of the activities being examined. Evaluation is a process of systematic inquiry directed at collecting, analyzing and interpreting information so that one can draw conclusions about the merit, worth, value or significance about a program, project, policy or whatever it is that is being examined. Outcome evaluation, then, at its most general level, is a systematic examination of the outcomes (changes, usually benefits), resulting from a set of activities implemented to achieve a stated goal, and a systematic examination of the extent to which those activities actually caused those outcomes to occur. The intent of outcome evaluation is to assess the effectiveness of these activities with respect to the benefits achieved, suggest improvements and possibly provide direction for future activities.’
And liked another writer’s graphic + example illustrating the relationships between outputs, outcomes and benefits which goes, ‘in a project designed to implement a document management system:
- The implemented document management system that users can actually use is the output
- This output results in certain outcomes, for example:
- faster and easier access to the documents
- fewer mistakes in archiving and retrieving documents
- ability to control versions and having access to the latest revisions
- ability to have reports that help us find solutions for other business problems
- And finally, those outcomes create benefits, or in other words, business value:
- 2% cut in the operation cost
Note that in this example the measurement comes at the benefits realisation stage and not at the outcomes stage – the ‘basket’ of outcomes yields the measurable benefit. However, in many cases outcomes are also measured e.g.
‘Health outcomes involve changes in health status – changes in health of an individual or population, attributable to an intervention. Sometimes the population or group is defined because different outcomes are expected for diverse people and conditions. Measurement of health outcomes involves carrying out different measurements including, measurement of health status before the intervention, measurement of the intervention, and measurement after to try and relate the change to the intervention.’ and from a different author ‘The healthcare industry must measure outcomes to identify which treatments are most effective and provide the most benefit to patients.’
Generally leaders/managers have difficulty in being clear in what outcomes and benefits they are looking for in organisation design work. (Saying ‘better teamwork’ or ‘cost reduction’ is insufficient). To help with this I showed participants a list of benefits that organisation design work could realise. I’ve found that discussing these helps identify and clarify the outcomes that leaders/managers are looking for. The list is as follows:
Strategic Fit: Benefits that contribute to the desired outcomes of strategic objectives or make them achievable.
Operating Model: Benefits that are derived from structural change, better resource management and decision making – e.g. more efficient centralisation, co-ordination and control of activities through clearly defined capabilities, roles, responsibilities. Performance improvement through re-engineered, automated processes and shared centres of expertise. A consistent system of new or amended policies, standards and working practices which enable investments in systems and tools to be leveraged at scale.
People: Benefits of a better motivated workforce e.g. flexible working and increased productivity through variety of work and career opportunities, professional support, training and development.
Quality of Service: Benefits to stakeholder groups, e.g. raising the customer experience through consolidation of volumes and services, standardisation of people assets and ways of working across many customers etc.
Budgetary Control: Benefits which allow for improved control through a framework within which the costs of introducing new infrastructure, standards and quality regimes can be justified, measured and assessed.
Risk reduction: Benefits which allow the organisation to be better prepared for future customer service provision, through greater access to a range of specialisms and expertise via a more skilled pool of professionals.
Economy: Benefits that deliver a lower cost of service whilst maintaining quality, resilience and flexibility.
From the above, it’s easy to believe that measurement of outcomes and benefits ‘can be simple’. For example, Jed Simms says it will be ‘if you just follow this 10-step process:
- Start with the end defined: Clearly identify your desired business outcomes, or where you want your business to be at the end of the project.
- Identify the benefits that come with achieving these desired business outcomes. Link each benefit to the outcome that will deliver it.’ And so on, up to 10.
But it’s not. Toby Lowe and Rob Wilson talk about ‘The paradox of outcomes—the more we measure, the less we understand.’ Their argument is made in relation to social policy outcomes, but the foundation for the argument is based in complexity theory.
Their paper Managing the performance of social interventions. What can we learn from a complex systems approach? Is well worth reading. If we believe we are in a VUCA world (volatile, uncertain, complex, ambiguous) then we should be willing to look at what we mean by ‘complex’, and be open to complexity approaches which, as they say, require us
‘to give up simple notions of cause and effect, and abandon reductionist approaches to understanding social phenomena. [Complexity] requires us to think differently about how we seek to make change in the world, accepting that the changes we desire are beyond our control. In Boulton et al’s (2015: 108/9) words: “to truly accept that the world is complex changes us. It fundamentally causes us to rethink how we approach the world, how we make sense of what happens, how we approach everything we do. … A key aspect of a complexity worldview is that the interaction of factors influencing a given context means that the future is irredeemably unpredictable. … in situations of complexity, cause and effect do not operate in linear fashion. … Non-linearity is hugely significant, because it means that what happens in situations of complexity is emergent: it is not predictable from the starting conditions of the situation.’
The linking of organisation design interventions to predictable outcomes and benefits is a linear approach where there is an assumption of cause and effect. What is does not take into account is the continuous flux and shift in the conditions around the design work – people moving roles, new priorities, sudden constraints, external demands, and so on. Even if carefully planned and executed our organisation design work is not happening in a stable vacuum – it is (usually) responding to a complex, emergent situation with multiple variables. As Lowe and Wilson say, ‘Outcomes, which may have been desirable at the start of a process may not be suitable for a context which has changed. And the opposite is also true: outcomes which were undesirable or unforeseen may come to be seen as crucial.’
This does not mean we shouldn’t attempt to measure. Lowe and Wilson offer ‘starting points for discussion in this field by identifying the purpose of a complexity-friendly performance measurement system, and elements of both a conceptual framework and mechanisms by which this might function.’ Their framework ‘requires a different way of thinking about some of [measurement] core concepts, particularly ideas of accountability, trust and the locus of key decision-making.’
Are you measuring outcomes and benefits? Is it useful to consider complexity approaches in this? Let me know.
Image: Complexity of Life, Rick Stevens 2014
Naomi Stanford 
Great read thank yoou