As public involvement has grown, so too has a desire to measure and demonstrate its impact. It’s important to have processes in place to help you reflect on public involvement throughout the research cycle, so that you can learn from the experience and act on that learning.
There are different types of impact that we need to measure relating to PCIE:
- the impact on research processes and outcomes
- the impact on participating members of the public and researchers
- the impact on addressing power imbalances between the researcher and the researched
We need good evidence to understand the difference that public involvement makes, but clear answers as to how best to measure this are hard to identify. Some researchers have designed tools for recording involvement, including the GRIPP Checklist (Guidance for Reporting Involvement of Patients and Public) and the PiiAF (Patient Involvement Impact Assessment Framework). These tools may help you to quantify public involvement in your project, but do they also capture the softer aspects and the complexity of public involvement, i.e. relationships and trust built, collaborations, networking etc?
Challenges with identifying and quantifying impact can include:
- Lack of agreement about what is meant by 'involvement and 'impact'
- Do we measure the process of involvement (numbers involved) or the difference it makes to the research outcomes?
- Is public involvement impact captured during the research process, may be difficult to identify after the event?
- Involvement can affect changes over a long period of time, through networks and connections, making it difficult to attribute changes to the original involvement
- Are we measuring the impact people were allowed to make? Or could have made? (barriers may include time constraints, finances, networks, flexibility of researcher)
Methods for assessing impact:
Quantitative:
- Numbers of people involved
- Numbers of focus groups
- Number of co-designed questionnaire or summaries
These types of metrics alone may not always capture the full impact of the public involvement, including the networks developed, the change of research focus, the power dynamics of a team or the empowering of people/communities.
Qualitative:
- The use of narratives (stories) of how things have changed
- Feedback to public contributors can be a record of change and intention at that point in the project
- Surveys to capture the public contributors experience
From the beginning of your research project, work with your public contributors to think about how you are going to monitor and evaluate public involvement and its impact throughout the project.
References and Links to Resources:
Patient and public involvement in research. The value and impact of patient and public involvement in research.
Russell, J., Fudge, N. & Greenhalgh, T. The impact of public involvement in health research: what are we measuring? Why are we measuring it? Should we stop measuring it? Res Involv Engagem 6, 63 (2020).
Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research BMJ 2017; 358: j3453 doi:10.1136/bmj. j3453
Get a GRIPP2! Top 5 things you need to know about reporting patient and public involvement in your research project. CLAHRC East of England. 2017
Popay J, Collins M, PiiAF Study Group. (2014). The Public Involvement Impact Assessment Framework Guidance. Liverpool and Exeter: Universities of Lancaster.
