PPI at stages of the systematic review
A systematic review is a process involving a series of different stages. The Cochrane review ecosystem illustrates 11 key stages of a systematic review, from developing the question through to writing and publishing the review. A final, 12th, stage is disseminating the results of the review. People can be involved at any (or all) of these stages. There should always be a clear aim or objective associated with involvement of people within a systematic review. Often the aim will relate to decisions that need to be made within the systematic review process. Depending on the aim of involvement, people may be involved at 1 stage, at 2 or more stages, or they can be involved throughout the whole review.
The Cochrane Involving People learning resource provides examples of systematic reviews that have involved people at the 12 different stages of a review process in order to meet a range of different aims. Table 2 provides some brief examples of PPI at different stages of systematic reviews, taken from the Involving People resource.
Table 2 Examples of involvement of people at different stages of systematic reviews (from the Cochrane Involving People learning resource)
| Stage of review | Example review (reference) | Aim of involvement | What happened? |
| 1. Develop question | Edwards et al. (2015) | Clarify the review questions in a systematic review relating to complex mental health needs and services for children and adolescents in the UK | Edwards et al. (2015) used 2 different strategies. In 1 strategy, 6 young people who had been mental health inpatients, were interviewed, individually. The aim was to identify topics for the review to focus on. In the second strategy, healthcare professionals, young people and charity representatives met face to face to generate and rank topics of importance. |
| 2. Plan methods | Pollock et al. (2015) | Clarify methods for a Cochrane review update relating to physiotherapy for people who had experienced a stroke, in particular the categorisation of interventions | Pollock et al. (2015) formed a stakeholder group of patients, carers and healthcare professionals. There were 2 meetings that focused on clarifying methods of the planned review. The stakeholder group’s input generated a method for categorising interventions within the review. This method was used to structure the final review and also informed subgroup analyses. |
| 3. Write and publish protocol | Liabo (2013) | Agree the protocol content for a review focused on interventions to support looked-after children in school | Liabo (2013) used a participatory approach to involve a group of young people throughout the review. At one of the meetings, participants were presented with a pre-prepared document with tick-box options for different alternatives within the protocol. The options had been generated from the discussions at previous meetings that focused on the review question. The text included in the final protocol reflected the views that had been collected during the tick-box exercise and associated discussion. |
| 4. Develop search | Rees et al. (2004) | Advise on terminology for the search strategy, for a systematic review relating to HIV-related sexual health for men | Rees et al. (2004) involved a range of people in 3 meetings. In one of the meetings, the group specifically advised on terminology for the search strategy. |
| 5. Run search | Harris et al. (2016) | Help identify unpublished papers, for a realist review relating to community-based peer support | Harris et al. (2016) established an advisory network of stakeholders. Communication with the advisory network occurred through a series of events, as well as less formal communication, including email. Harris et al. report that advisory network members helped to identify relevant unpublished papers during the searching phase. |
| 6. Select studies | Vale et al. (2012) | Provide oversight to a Cochrane review of chemoradiotherapy for cervical cancer | Vale et al. (2012) formed a group of ‘patient research partners’ who provided continuous oversight for the review. The group was actively involved in several review tasks, including tracing the address details of trial investigators for studies selected for inclusion. |
| 7. Collect data | Bayliss et al. (2016) | Co-produce a coding framework for the qualitative analysis in a qualitative systematic review focused on predictive testing for those at risk of developing a chronic inflammatory disease | Bayliss et al. (2016) had a group of ‘patient research partners’ who provided continuous oversight for the review. Three of the patient research partners volunteered to be involved in the qualitative analysis. They coded themes for a random selection of 3 papers and contributed to developing a co-produced coding framework in collaboration with the researchers. This was done through email correspondence. Written training documents were developed to support the volunteers with this involvement. |
| 8. Assess risk of bias | – | – | There is little evidence of involvement of stakeholders in the process of assessing risk of bias. Liabo (2013) reported that ‘none of the young people were interested in being involved in activities that required them to read the full studies’. As a result of this observation, these stakeholders were involved in ‘a general discussion about research quality rather than aiming for them to take an active part in reading the studies and assessing them for quality’. |
| 9. Analyse data | Bayliss et al. (2016) | Consider and comment on the qualitative themes generated for the qualitative synthesis | The patient research partners involved in the review of Bayliss et al. (2016) attended a face-to-face meeting to which all stakeholders were invited. They read all the included papers before the meeting. Bayliss et al. reported that this session aimed to help researchers draw on the perspectives of the patient research partners when interpreting and reflecting upon the data. |
| 10. Interpret findings | Pollock et al. (2014, 2015) | Gain consensus on the clinical implications arising from the review | Pollock et al. (2014, 2015)held a stakeholder meeting at which the draft findings (results of meta-analyses) were presented. Stakeholders were asked to discuss the clinical implications of these findings. Through discussion, the group agreed the wording of a series of statements relating to clinical implications, with anonymous voting used to confirm agreement with the statements. The agreed statements were included with the published review. |
| 11. Write review | Concannon et al. (2014) | Get feedback on drafts of a systematic review of methods of stakeholder engagement in research | Colcannon et al. (2014) held 2 face-to-face meetings with a group of stakeholders, who also participated by email and phone throughout the review process, including commenting on tables, figures and manuscript drafts. Colcannon et al. stated that ‘stakeholders [at a second meeting] also helped us identify effective ways to communicate the findings in tables and figures for this manuscript. All stakeholders were invited to participate by email and phone throughout the research, including a review of the manuscript’’. |
| 12. Publishing the review and disseminating | Hyde et al. (2017) | Plan and contribute to disseminating the results of a review focused on factors affecting shared decision making around prescribing analgesia for musculoskeletal pain | Hyde et al. (2017) held 3 stakeholder meetings at different stages during the review. Group members were involved in ‘planning how to share results’ and ‘agreeing dissemination of the results’. Consequently, ‘results were targeted at practitioners, as [stakeholders] felt this was most important’. Hyde et al. reported that group members ‘participated in dissemination of the review findings’. They also reported that they ‘planned their own roles…including giving presentations and contributing the patient’s perspective to discussions at conferences’. |
Top and tail approach
Pollock et al. (2019) explored when systematic review authors had PPI in their reviews. They found that people were most commonly involved at the initial stages (stages 1 to 3: framing the question and planning the review) and the final stages (stages 10 to 12: interpretation, publication and dissemination of findings). It was less common for people to be involved during the middle stages (stages 4 to 9: conducting the review). Often people were involved at both the initial and final stages, but not in the middle – this has been termed a ‘top and tail’ approach (Pollock et al. 2019). A top and tail approach may involve the same group of people at the start and end of the review, or it may involve 2 different sets of people.
For systematic reviews being planned and conducted as part of the development of a guideline, a top and tail approach could potentially fit efficiently within the guideline process. However, there is no evidence to support this as being the ‘best’ approach, and decisions about when to involve people should be made based on the pre-determined aims of involvement for each individual systematic review.