The results of the VOICES consultation directly affected the European Commission’s Horizon 2020 calls for research funding. As such, VOICES is a model for incorporating citizens’ voices into RRI. But the project outcomes can also be used by other stakeholders on a local and national, as well as European level, for example:

• Industry can make use of the results to look into gaps in the market for new innovations;
• Researchers can use VOICES results to align their research with the principles of RRI;
• Educators can use project outcomes to give a national and European perspective to classroom discussions on current science topics;
• Universities can use VOICES as a model for citizen participation;
• Policymakers and funding agencies both within and beyond the European institutions can shape their research strategy to fit RRI.

For practical information on how to implement a consultation like VOICES, download the following documents.

About the methodology

As a qualitative research method, focus groups are increasingly used in political and social sciences, and can be defined as "a carefully planned discussion designed to obtain perceptions on a defined area of interest in a permissive, non-threatening environment".

An important advantage of focus groups in comparison to other research methods is that participants can respond to and build on the views expressed by the other participants. Because of this interaction, focus groups generate a large variety of opinions and ideas which provide insightful information, while maintaining a specific focus during the discussion.

The method provides the opportunity to gain in-depth insight into ideas, values, wishes and concerns of participants and stimulates shared creative thinking. A specific characteristic of the focus group method is that it seeks understanding of a research topic from a particular perspective; in the case of the VOICES project, the perspective of European citizens.

VOICES methodology - [PDF - 9.5Mb]

Moderators working group

One moderator from each of the 33 locations holding VOICES focus groups attended the VOICES working group session in Brussels from 6th to 8th March 2013. During the intensive course, moderators became familiar with the VOICES methodology and practiced the exercises foreseen in the script.

VOICES practicalities - [PDF - 7.2Mb]
Each moderator received a copy of the Moderators manual [PDF - 2.4Mb].

The training focused on specificities of the VOICES focus group script as well as on refining important competencies of the moderators' role, including interpersonal communication, process management and understanding of the topic addressed.

VOICES approach to urban waste - video presentation by Frank Kupper, Senior Researcher, Athena Institute, VU University Amsterdam

Focus Group organisations - video presentation by Marzia Mazzonetto, Senior Project Officer, Ecsite

Role of Moderators - interview with Jacqueline Broerse, Athena Institute, VU University Amsterdam

Selection of focus group participants

Participants were selected by local recruitment agencies, according to predefined selection criteria. Selection criteria were applied in order to obtain diversity in focus group participants, and to represent society at large.

General selection criteria with respect to demographic information included sex, education and employment. The focus groups were stratified by age using the following categories: 18 to 35 years of age, 36 to 50 years of age and 50+.

Other criteria addressed elements relevant to the VOICES project's specific topic, including: participants from urban and non-urban areas, diversity of types of municipality and diversity of housing situation. These selection criteria were applied in all EU member states.

Focus Group Structure

The focus groups lasted 3 hours and followed a semi-structured script [PDF - 125Kb] consisting of an introduction, four main exercises and an evaluation part. Each focus group was lead by a moderator, who was in charge of stimulating and guiding the discussion. The moderator's role was also to maintain the focus of the discussion by ensuring that key themes were covered, while managing group dynamics.

Interview with Luca Simeone, Project Manager, Città della Scienza

In order to capture the data generated during the process, audio and/or video recordings were made of all focus groups. A note taker was also required to be present for the entire duration of the focus groups, in order to record additional data and to assist the moderator.

All visual data generated by the participants, for example, individual drawings or collective mind maps, were collected at the end of each focus group and photographed.

Participants' reactions to focus groups

Ideas, suggestions and interest expressed during VOICES focus groups - Interview with Luisa Marino, Project manager, Ecsite

Analysis

After each focus group, a summary report was written by the moderators based on the note taker's notes and the information on the flip charts. A draft of this summary report was sent to the focus group participants who were asked to comment on it.

Moderators collected any feedback and included it in the final version of the summary report as an annex.

The audio recording of each focus group was transcribed word-for-word and translated into English for analysis.

The translated transcripts were coded and analysed using MaxQDA, a programme for qualitative data analysis.

For the analysis of the data, both structured analysis as well as open coding were used. Structured analysis was carried out by using a predesigned coding sheet based on preliminary research.

This type of analysis allows for all relevant outcomes to be extracted from the raw data. Open coding runs parallel to the structured analysis and allows for insights unforeseen by preliminary research to emerge.

The summary reports of the individual focus groups were used to validate and complement the analysis.

See VOICES Reports for the results and analyses in full.

Validation, Consolidation and Evaluation

VOICES benefitted from a great deal of expert input, including:

• The VOICES Focus Group approach was validated by the VOICES Advisory Group, composed of 10 experts in public participation, RRI, urban waste/environment issues, social innovation, governance, qualitative methodologies in social research and the science museum field.
• A Consolidation Group, from research and academia, business/industry/SMEs, public authorities, civil society and non-government organisations, and other key stakeholders, who assessed the outcomes of the focus groups and provided the European Commission with advice and expertise in order to transform citizens' ideas into priorities for the 2014 work programme on 'Urban Waste'.
• A full independent evaluation assessed the quality of the project and the participatory processes providing suggestions for improvements in the future;
  
  For more details on the external experts involved, see the VOICES project page.