Thursday, July 20, 2017

Case Study: Exploratory research project on the 1290 expulsion of the Jews from England for the Migration Museum Project

Introduction

The Migration Museum Project (MMP) are planning a new London-based exhibition in September 2017 called “No Turning Back.” The UK charity, which aims to create a museum on migration for Britain, is working with volunteer researchers on six different moments of significance in Britain’s migration past and present to build their knowledge of these moments and develop a public exhibit that is accessible to all ages and a range of audiences.

OPM Group’s Corporate Responsibility Working Group (CRWG) volunteered to contribute to this exploratory research with the MMP. Research on one moment, “The 1290 Expulsion of the Jews from England” began in March 2017 and was completed in June 2017.

 

Methodology

OPM Group provided a team of eight volunteer researchers to gather data and manage the collection of facts, images and stories relating to one of six moments the MMP will feature in “No Turning Back”. For the research, we also identified key artists and experts for the MMP to gain additional insight and resources. Volunteer researchers used Google searching and contacts established through the MMP to develop an initial scoping of extant information on the moment.

We then wrote an interim report for the MMP and received guidance on areas for further exploration from the its research and curatorial leads. Volunteer researchers completed additional research on the moment and a final report was submitted to the MMP in June 2017.

 

Impact

Our detailed and accessible report has allowed the MMProject to incorporate an exhibit on the 1290 expulsion of the Jews because of the information we collected. The MMP is pleased with the result of this voluntary work:

Thank you so much for all your hard work on our account and for your beautifully presented and detailed document. It has helped us a great deal, saved us a huge amount of time and we would never have managed this without you. I hope we can do you justice in the final exhibition.” – Museum Curator.

Friday, May 12, 2017

Bake My Day!

I recently discovered a new facilitation tool. Bread making. When in doubt, if you’ve got a tricky subject matter, or disparate group of people, bake a loaf.

As part of Marmalade 2017, Arts at the Old Fire Station, Camerados, and Mayday Trust hosted a workshop called Bread and Butter Services. This workshop intended to explore the value of relationships in addressing problems caused by isolation and loneliness. There were about 45 participants; a mixture of organisations providing services for homeless people, service commissioners, and people with lived experiences of homelessness and times of crisis.

You can watch a film about the whole day here.

OPM Group’s “Dialogue by Design” team supported the design of the event, and facilitated the day. Aside from the endless supply of fantastic(ally awful) puns that come with bread baking as a workshop activity, there are a host of reasons why it really works. Here are my top 5:

1) It gives people something to do other than talk to each other. This may seem an odd thing to say when often successful workshops are built on the quality of the conversations that take place. However, sitting across a table from someone else, aware that you need to reach some sort of outcome by a certain time of the day, can produce a very forced conversation. This is especially true when working with a group of people who may find it difficult to interact with each other. Giving people an activity to do together takes the pressure off and allows people to interact more naturally. The conversations that need to happen can still happen, but in a much more relaxed way.

2) It builds trust. Providing an activity that has nothing to do with the subject matter of the workshop encourages people to see each other as people – not as their job titles. Power dynamics and tensions in the room quickly diffuse as people come together over a simple, fun activity, in which everyone can easily participate. As a result, conversations become more human, more honest, and more productive.

3) It introduces a little chaos. Not everyone is comfortable with highly formal, organised processes. While other elements of the day were more standard design-workshop style activities, the bread-making ensured there was always an element of unpredictability running throughout. This was reassuring for those for whom a workshop or conference-style environment was new and intimidating, and conversely was stimulating for those who may have been dreading the standard flip-chart and post-it-note workshop routine.

4) It doesn’t take over the day. At first, I did think we may have bitten off more bap than we could chew by trying to get to the end of the day with solid workshop outputs AND edible bread products. However, bread baking can really be timed around the other activities, and actually doesn’t take too long. Our participants probably spent a total of an hour on bread-related activities, and the time that was spent doing that was invaluable in terms of ensuring points 1 and 2 above happened early on in the day.

5) You can eat the output of your workshop at the end. Once we had finished for the day we brought in the baked loaves, with some jam and cheese and drinks, and invited everyone to enjoy what they had made together. This provided not just a great metaphor for collaboration and building positive relationships, but also facilitated exactly that.

The event was well received by all participants. Seven subject-specific outcomes were developed during the day, as well as five key behaviours to embrace (for more information see the event report produced by the Arts at Old Fire Station and this blog post from Lankelly Chase)- so the bread was certainly not the only positive product of the event. For more information about Marmalade, please get in touch with Arts at the Old Fire Station – and check out the video wrap up for this year. For information about the process design for the workshop, (bread making and otherwise) contact anna@dialoguebydesign.co.uk

Anna McKeon
Consultant
Dialogue by Design

 

 

Monday, March 20, 2017

Commissioning for Outcomes – The role of social finance

Can social finance help with the challenges that public commissioning faces?

This paper is intended as a provocation to government, commissioners, providers and investors to begin a richer conversation that doesn’t assume we already know the answers. OPM’s experts in commissioning for outcomes (Sue Goss) and in social finance (Chih Hoong) draw on their learning about systems leadership, experience of teaching commissioning programmes and our work in evaluating social investment experiments.

Monday, September 19, 2016

Southend Vision for 2030 – Southend on Sea Borough Council

Background

Following an initial phase of community and stakeholder engagement carried out by Southend-on-Sea Borough Council, the OPM Group was commissioned by the council to deliver a community engagement programme to support the Borough and Council in setting its community vision. The aim of the project was to facilitate a series of conversations to explore and develop the collective aspirations of a different groups within the community for the Borough, as well as identifying how these groups will work together to achieve it against a backdrop of reducing resources.

What did we do

In order to ensure that conversations were grounded in a detailed understanding and appreciation of the challenges and opportunities facing the borough, the initial stage of the project centred on in-depth research and scoping. This comprised community and stakeholder mapping, a rapid document review of recent public engagement and service monitoring data for the borough and a series of telephone scoping interviews.

The central element of our approach focused on a participation and engagement, harnessing the energy and enthusiasm of communities to ensure buy-in to the vision being developed. We carried out a series of workshops involving a wide cross-section of Southend’s communities, from businesses to grass-roots local organisations to residents more widely.

Outcome

These events comprised a smaller, targeted focus-group style discussions with specific groups as well as larger open-invitation workshops and enable local people to contribute thoughts and ideas to the ‘narrative’ being developed so that it reflected a genuine community-wide, collaborative discussion.

Monday, August 15, 2016

Handle with care: a case study in using baseline and follow up surveys

The new orthodoxy

Baseline and follow up methodologies have become the new orthodoxy when assessing the impact of interventions on programme participants – particularly when evaluating programmes with children and young people. In a context where experimental approaches are rarely seen as workable or even ethical, collecting baseline and follow up data is increasingly the default option expected by funders and commissioners. They are also relatively cheap to deliver – which, in current times – is appealing.

The evaluator will typically convert a programme’s anticipated outcomes into a set of indicators, use these to form the basis of a survey, and then invite participants to complete the survey at the start and end of the programme. They then hope for the best, based on an assumption that the programme will see a positive shift against the indicators over the course of the intervention.

However, when the results come back there are not always neat improvements in the follow-up. This is where disappointment and panic can set in. A surface level analysis of the data might suggest that the programme has had no – or indeed had a negative – impact on participants. This is not good for busy practitioners who are juggling delivery with the pressure of proving the impact of their work, particularly when they rely on good outcomes for future funding!

Drawing on our recent experience of evaluating a range of complex social interventions, while baseline and follow-up approaches are not necessarily the wrong tool for the job the results need to be carefully interpreted and contextualised and – where possible – cross-checked against other strands of evidence. Only then can you begin to draw out some sensible conclusions about the impact that might have been achieved.

A case study: evaluating Body & Soul Beyond Boundaries programme

We recently conducted an impact evaluation of Beyond Boundaries, an innovative peer mentoring programme for HIV charity Body & Soul. This used a baseline and follow up survey against key indicators. When the results came in there was a positive progression against the majority of the indicators. Young people who had been on the programme were more likely to communicate openly and honestly about their health and their HIV status by the end of the programme. However respondents responded less positively to some wellbeing indicators around self-respect and general state of happiness.

It would have been easy to assume that the programme had had little impact on these areas of wellbeing. However, further scrutinising of the different strands of data allowed us to develop an alternative analysis.

1) Beyond Boundaries did not operate in isolation from other factors that influenced participants’ lives. Programme staff emphasised that the particular cohort they are working with led very chaotic lives and tend to experience a myriad of socioeconomic issues. Here it could be reasonably argued that their participation in the programme may have been helping them to maintain a certain level of wellbeing, and possibly even prevented a more severe decline against these indicators.

2) As a consequence of receiving support and building trust with the service, there are examples where participants increased their emotional literacy and become more willing and able to answer openly and honestly about how they feel. This could explain how they became more able to appraise their personal sense of wellbeing in a more critical way. This finding is consistent with wider evidence that suggests that young people in particular are likely to overstate their levels of wellbeing at baseline and then provide more open and critical score in the follow up.

A further challenge was that, for a whole host of reasons, there was a high rate of attrition between participants completing the baseline and the follow up. This meant that the survey data in isolation did not produce a robust understanding of the impact of the programme. However, when this data was taken in combination with other strands of the evaluation it was possible to make stronger claims about the impact. Triangulating the findings from the baseline and follow up survey with case study and focus group data also allowed us to better understand participant’s journeys and to explore the impact of the programme on the most vulnerable and hard to reach participants, who are often the least willing to take part in evaluation activities.

Focus on why and how, not before and after

Collecting detailed qualitative feedback from programme staff and participants helped us to explore those crucial “why?” and the “how?” questions which helped to shed light on the pathways to outcomes. This was crucial when exploring the impact of a complex service used by participants who have equally complex lives.

Tuesday, August 9, 2016

Introducing our new blog series – engaging children and young people in research and evaluation

At OPM we jump at the opportunity to work on projects involving children and young people. Not only are they tremendously fun to work with, but we passionately believe in the importance of their voices being heard in issues that affect them.

From researching young people’s views on police stop and search powers to evaluating their experiences of major government policies like the SEN and disability pathfinders, I have been privileged to meet some amazing and inspiring young people in the course of my work here. And we are constantly developing our practice in this area to make sure our research combines meaningful participation by young people, with compelling evidence generated for our clients.

OPM Group colleagues have written a series of thoughtful blogs on this topic. First we have two linked pieces on challenges around measuring impact – both highlighting examples of our recent work with young people, but with plenty of relevance for research with adults too.

Tim Vanson’s blog draws on a recent impact evaluation of an innovative peer mentoring programme for HIV charity Body & Soul to explore the current emphasis on baseline and follow up methodologies, and the challenges of using surveys in this way to demonstrate impact. When surveys don’t show the results you might expect, how do you make sense of this and present the findings so that they actually tell us something meaningful?

Caitilin McMillan and Bethan Peach focus on soft skills, which many programmes and interventions aim to improve, particularly those involving children and young people. And many succeed in doing so, but it’s notoriously hard to prove this with any accuracy or consistency. In OPM’s evaluation of the GLA’s Stepping Stones programme, we are taking a fresh approach to this using the ‘virtual young Londoner’ tool. Read about how this enables participants to speak more openly about their aspirations, behaviour and attitudes, without feeling overly vulnerable or exposed – especially when speaking in peer groups.

Finally, Bethan Peach shares her 7 top tips for conducting qualitative research with children and young people, based on her recent experiences during a project for Essex County Council evaluating the impact of early help for families, children and young people. A sneak peek: don’t assume you know what’s going to work with any given group of young people, because they’ll probably surprise you. So be prepared for anything to happen, and enjoy when it does!

If you’d like to get in touch to discuss any of this work, or the other research we have done with children and young people, then drop me an email.

Tuesday, August 9, 2016

Our work with children and young people is informing best practice

We are delighted that our evaluation of the vInspired 24/24 programme has been included in the systematic review of the Early Intervention Foundation (EIF) Social and Emotional Learning Programme Assessment, contributing to the evidence base of best practice in this area.

The EIF has been funded by the Joseph Rowntree Foundation to undertake a programme of work to understand what works in developing the relational, social and emotional capabilities and assets of children and families either in or at risk of poverty.  This work builds upon and extends an existing review carried out by EIF in 2015: ‘Social and Emotional Learning: Skills for life and work’

The 24/24 programme, managed by vInspired and funded by the Department of Education (DfE) and the Jack Petchey Foundation, was a structured volunteering and social action intervention programme, designed to help young people facing challenging circumstances to improve their life choices. The evaluation was designed to measure the progress of individuals during the programme and evidence the impact of the programme on young peoples’ outcomes and against key performance indicators. It also identified the critical success factors and limitations of the delivery model and made recommendations for future delivery.

The evaluation report can be accessed here.

Tuesday, June 7, 2016

New evaluation of STEM Learning Triple Science Support Programme

We have just completed a two-year evaluation project for STEM Learning (a national body providing schools with continuous professional development in science, technology, engineering, and maths), looking into the impact of the Triple Science Support Programme (TSSP) in 2014-2016. The TSSP supports schools to develop their triple science provision at GCSE level. In our evaluation we spoke to teachers and delivery staff, and found that the TSSP has helped schools make considerable improvements in their triple science provision. We found the project had a range of positive impacts, including raising subject teachers’ confidence; improving science departments’ capacity and capability to teach triple science; and outcomes for students such as improved motivation, and progress and attainment in triple science. Download the full report here.

Tuesday, May 17, 2016

Evaluation of the Reducing Social Isolation and Loneliness Grant Programme

Social isolation and loneliness in older people is a widespread issue that has gained much attention in recent years. We know that being isolated and lonely can impact on a person’s quality of life and lead to more intensive use of health and social care services.  

In Manchester the three Clinical Commissioning Groups provided grant funding targeted to reduce social isolation and loneliness amongst Manchester residents aged 50+. Grants were awarded to voluntary sector organisations to deliver 27 projects across the city. The Programme ran from September 2014 until March 2016 and was managed by Manchester Community Central (Macc).

OPM was commissioned to evaluate the Grant Programme. The evaluation sought to demonstrate outcomes and provide evidence around ‘what works and why’.

This presentation was delivered at the final Programme celebration event attended by representatives from the CCGs, other North West CCGs, Manchester City Council, Macc, local research organisations, plus VCS leads and volunteers from across the city. It presents the headline findings from the evaluation and showcases two projects in depth. Our final evaluation report will be available in the coming months.

Click here to download the presentation slides.

Tuesday, May 17, 2016

Evaluation of the Reducing Social Isolation and Loneliness Grant Programme

Social isolation and loneliness in older people is a widespread issue that has gained much attention in recent years. We know that being isolated and lonely can impact on a person’s quality of life and lead to more intensive use of health and social care services.

In Manchester the three Clinical Commissioning Groups provided grant funding targeted to reduce social isolation and loneliness amongst Manchester residents aged 50+. Grants were awarded to voluntary sector organisations to deliver 27 projects across the city. The Programme ran from September 2014 until March 2016 and was managed by Manchester Community Central (Macc).

OPM was commissioned to evaluated the Grant Programme. The evaluation sought to demonstrate outcomes and provide evidence around ‘what works and why’.

This presentation was delivered at the final Programme celebration event attended by representatives from the CCGs, other North West CCGs, Manchester City Council, Macc, local research organisations, plus VCS leads and volunteers from across the city. It presents the headline findings from the evaluation and showcases two projects in depth. Our final evaluation report will be available in the coming months.