News and Comment

Practical tools to help demonstrate impact and value

Tuesday 20 September 2011

By:

By Dr Chih Hoong Sin, OPM principal and Shelley Dorrans, OPM fellow

We recently attended the National Commissioning and Contracting Conference and ran a workshop on practical hints and tools that can help service providers demonstrate impact and value. The presentation also aimed to help commissioners understand better what they should ask providers to demonstrate. The workshop was oversubscribed, perhaps testament to the growing demand in the field for practical tips in an area often regarded as difficult and confusing, yet increasingly vital.

We know from published evidence and from our engagement with commissioners and providers that they face real challenges. Commissioners can struggle to articulate what impact providers are required to demonstrate. For example, the debates on Payment By Results attest to concerns that ‘results’ may be pegged to ‘outputs’ rather than ‘outcomes’. Workshop attendees raised this concern. Outputs are often easier to measure than outcomes, and there is a risk that the effectiveness may be judged against what is easy to measure rather than what is important.

At the same time, we also know that providers can often perceive monitoring returns prescribed by commissioners to be irrelevant, meaningless and burdensome. Monitoring and evaluation can be regarded as paper exercises to satisfy the needs of funders rather than making any real difference to the work they do and the people they serve. For example, while the Coalition Government encourages the voluntary and community sector (VCS) to be involved in public service delivery, Charities Evaluation Services reported that the information practices of the VCS are increasingly driven by compliance and reporting. Much of the sector does not feel that they are using data to improve organisational learning and ongoing service delivery.

Changing mindsets

From our experience of working with providers and commissioners, we realise the importance of changing mindsets on the practice of demonstrating impact and value. Rather than to be treated merely as a set of administrative procedures, we encourage understanding of key principles that can help people plan, do and review (commission) more effectively as part of routine practice. These are supported by simple frameworks and tools to help people understand these agendas, and that they do not need to be measurement experts.

Pathways to outcomes

Some of the tools we have designed, which were presented at the conference, include our ‘pathways to outcomes’ tool to help commissioners and providers visualise services in a whole-system, outcomes-focused way. This visual tool is accompanied by short guidance that helps users develop their ‘pathways to outcomes’ models individually and collectively. It prompts users to think systematically about five key questions at critical stages of a service model or a service user journey. It is designed to help clarify key implementation processes, structures, partners, beneficiaries, intended and unintended outcomes (short-, medium- and long-term), rationale and resourcing.

Demonstrating impact and value is not about collecting lots of data. It’s about having a clear framework for knowing what data is needed and why, and how it can best be collected and collated.

In the current age of austerity, the ‘pathways to outcomes’ approach offers an integrated framework for helping service commissioners and providers understand and articulate economic value in a way that is compliant with HM Treasury guidance on good practice. It helps users think through what they mean by ‘value’ and what the people they are trying to influence or persuade think of as ‘value’. It makes an important point that there is no one way of demonstrating value.

We must avoid Pavlovian responses of assuming that we all need to do cost benefit analyses, or that all VCS organisations should determine social return on investment. While these approaches certainly have their place, they can be resource and time-intensive and are often beyond the capacity of smaller organisations. We have produced a simple guidance document that introduces users to a range of different techniques, explains what each is designed to do and then takes users through a series of questions to help them develop a better understanding of what specific technique is most appropriate for their needs.

The focus on effectiveness, efficiency and quality will not go away. All service providers and commissioners need to get better at demonstrating impact and value. This is part of effective service delivery and commissioning, and should not be treated as a ‘bolt on’. It is not simply a measurement issue, but also about effective communication if the evidence is to be regarded as meaningful and impactful.

If you would like to learn more about our practical tools that can help you demonstrate impact and value, please contact us on csin@www.opm.co.uk or sdorrans@www.opm.co.uk.