Engaging with hard-to-reach and potentially vulnerable participants in the AP4L Project

By Dr. Ryan Gibson and Prof. Wendy Moncur

Introduction

AP4L’s overarching goal is to develop privacy-enhancing technologies (PETs) that support people undergoing a significant life transition to be safer online. The first step in building PETs that make a real difference is to understand what these people do online and the subsequent challenges they face. Central to this process is the involvement of participants from the four populations we are exploring in AP4L - Leaving the Armed Forces; LGBTQIA+; Living with Cancer; Relationship Breakdown – who can share their lived experience of transitioning online. 

These experiences may then be translated into design requirements for the development team to generate PETs that more accurately reflect the transition activities being conducted online and the related privacy risks and harms that people encounter. 

Nevertheless, designing and recruiting for research involving people who have undergone a life transition is extremely difficult. For example, protocols will centre on sensitive and possibly distressing times within an individual’s life that potential participants may feel uncomfortable reliving/discussing without an established relationship with the research team. 

This blog post provides insights into the methods used to maximise engagement with our target populations, along with the barriers we encountered with the survey. 

Survey Design

The study consisted of an exploratory, mixed-methods survey to build an initial understanding of participants’ online transition experiences. Questions were split into five main topics focusing on:

(1)    their demographics;

(2)    the nature of their life transition;

(3)    online activities related to this transition;

(4)    their technological experience; and

(5)    their psychological profile.

Unfortunately, we were not able to receive input from the PIP panel [1] (see previous blog) when designing the survey since it had yet to be established at that point. Nevertheless, great care was taken to ensure the open-ended questions explored the participants’ life transition in an ethically sensitive manner, using language accessible to individuals with a range of digital literacy skills. The survey length was also considered since some of our target populations (such as those living with cancer) may have had physical limitations that made it difficult for them to engage in lengthy research activities. Finally, due to the delicate and personal focus of the survey, it was important to establish an appropriate care programme for the participants. This involved making the participants aware of their rights (e.g. the option to skip any question, withdraw consent at any time, and access the data we hold on them) before the study to limit the amount of stress placed on them, alongside directing them to relevant support services in case they experienced distress at any time. These services consisted of more general organisations such as Samaritans, and transition-specific organisations like Maggie’s - who are involved as project partners and offered to support those participating in our studies. We piloted the final protocol with a few participants from our personal networks who had experienced at least one of the target life transitions to ensure the protocol was appropriate.

Survey Recruitment

Our target n-size for the survey was 100 participants per transition group to make concrete statistical conclusions from the collected data. Therefore, we decided to facilitate recruitment via an online research service. We felt it would be almost impossible to distribute the survey to 400 potentially vulnerable participants using traditional means such as social media posts. ‘Prolific’ was identified as our preferred survey distribution platform since it screened potential participants to ensure they fit our needs, e.g. location=UK; LGBTQIA+=yes; Chronic condition/illness=cancers. 

Four separate versions of the survey were developed for ‘Prolific’ to ensure we could recruit 100 participants per transition group. Within 48 hours of the surveys being published on ‘Prolific’, we received 100 completed survey responses from the LGBTQIA+, Living with Cancer, and Relationship Breakdown populations, which were subsequently reviewed for completeness by the research team. Any responses that had issues, such as failing the attention check questions embedded within the survey, were returned to ‘Prolific’ to allow further responses to be collected. We received 63 complete responses for the Leaving Armed Forces population over the first two weeks of data collection. Subsequently, we opened up the survey to American veterans to meet the target of 100, which was achieved soon after.

Observed Advantages and Disadvantages of Prolific

‘Prolific’ provided quick access to participants that we originally anticipated as hard to reach. The speed may be attributed to the anonymous nature of the platform and its use as a secondary income source by members of the public. Nevertheless, there were some downsides to its use. First, it is a relatively expensive service, costing £1120 inclusive of tax, to recruit 400 participants.  Second, due to the nature of the recruitment, the participants likely have above-average digital skills that may not be representative of their transition groups. 

This was confirmed by participants’ responses to the digital literacy questions within the survey. Finally, the qualitative data generated by the open-ended questions was not extensive and may have been a result of the aforementioned digital literacy that limits participants’ experiences with online harm. In addition, the participants may have rushed to completion, where they took an average of 15 minutes to record their responses, compared to the 30 minutes proposed by the pilot participants.

Conclusion

This blog post has discussed the importance of involving hard to reach and potentially vulnerable populations within research to ensure outcomes better meet their complex needs. In addition, we have introduced the methods used in the AP4L project, specifically an online survey, to enhance engagement with four life transition groups - Leaving the Armed Forces; LGBTQIA+; Living with Cancer; Relationship Breakdown – to support other researchers in working with these populations. In a future blog post, we will discuss the design of a series of 'creative security workshops' that built on the findings of the survey.

---

[1] Public Protection in privacy panel comprised of eight members of the public who have undergone at least one of the target transitions. Their role is to steer the research by utilising their own life experiences to provide advice on aspects such as research direction, study protocols, data interpretations etc.