A look under the surface of a public consultation: Part 4/5 – Reporting
The reporting process of a public consultation is the final culmination of all the stages we have previously discussed in this series of blog posts. During report writing, the views of all respondents are condensed into a document that is often available to the public, which represents a definitive output of the consultation process, along with the analysed data. This final output must accurately depict the full range and scope of the opinions raised in the consultation and effectively convey nuance and meaning in the key themes and arguments identified in the analysis stage.
For this reason we at Dialogue by Design take our role of safeguarding the quality and the integrity of the report very seriously. Our duty is not just to the client in providing the information they require, but to the consultation’s respondents too, specifically in demonstrating that their perspectives are captured and understood.
One of the guiding principles behind the HFEA report on techniques to avoid mitochondrial disease was that it should represent the range of views held by respondents, rather than simply focusing on the most frequently discussed themes. The report therefore covers themes and viewpoints raised by hundreds of respondents as well as those raised by a handful. This is because the purpose of this consultation – and others like it – is to uncover the whole range of public views on this subject and allow those with a particular stake or interest in the proposed techniques to have their say.
The report reminds readers that the consultation was held as part of a wider engagement programme including a public dialogue and research, and that some of these strands sought to obtain information about the range of views the general public holds on the subject. In this sense this consultation was very deliberately ‘open’ to all those who wanted to respond, in contrast to ‘sampled’ research which seeks the views of a statistically representative population sample.
In our consultation reports we always aim to strike a balance by indicating, on the one hand, where a view is held by a greater or smaller number of respondents, while on the other acknowledging that quantitative information of this kind is no measure for the balance of opinion held in the wider population. For this reason we don’t use percentages when reporting on findings from the analysis of consultation responses.
Considerations about reporting on quantitative information – e.g. how many respondents indicate that they agree or disagree with proposed new techniques to avoid mitochondrial disease – are especially important in consultations on controversial topics, as these often inspire organisations to mobilise their supporters to respond to the consultation in a particular way. We talk about ‘organised responses’ or ‘campaign responses’ when we see numerous consultation responses, submitted by different respondents, which all use the same words to convey a particular viewpoint or set of arguments. When reporting we make sure that what is being said in organised responses gets fair coverage, while also informing the reader of their ‘organised’ appearance.
This is not because Dialogue by Design wants to express an opinion on whether campaign responses are more or less valuable than individual responses, but rather for the sake of transparency, particularly with regard to numbers in the report. It is then up to the reader to decide how important the quantitative information is to them. Representing a mix of responses, from very detailed considerations by statutory stakeholders to snappy arguments echoed in hundreds of organised responses, is one of the main challenges thrown up by the reporting process. We believe it is more an art than a science to make sure that the final report is accurate and fair to all participants.
This is the fourth blog in a five-part series which analyses how Dialogue by Design, (part of the OPM Group), consults and reports on contentious subjects, paying special attention to a recent consultation conducted on behalf of the Human Fertilisation and Embryology Authority (HFEA).
‘A look under the surface of a public consultation’ blog series: