The Consumer Insight Panel was set up in September 2021 and compliments Which? 's existing quantitative Consumer Insight Tracker survey. During the year-long pilot we are conducting four quarterly semi-structured interviews by online-video and a number of online ad hoc projects with each participant. We have recently completed the 3rd wave of interviews.
Choosing the right sample
One of the first steps to setting up a panel is to consider the reach, or who you need to recruit. For this longitudinal panel, we recruited 35 UK households that reflect a broad spectrum of the UK population. There was a slight skew to those outside of London and on lower incomes. Due to the online methodologies used in this study there is also a slight skew to those who are considered as more tech-savvy or literate. We decided on a sample with this make-up for two reasons:
- Differentiation - as part of the panel we wanted to ensure we reach participants who we are not yet reaching through other data sources e.g. our member panels, mystery shopping, and quantitative methods. It is important to ensure new data sources compliment existing ones rather than conflict.
- Complimentary - one of the topics we aimed to explore within the panel is household income, particularly how it changes across the course of the year and how households adapt to these changes. As a result of this we wanted to ensure we are capturing the experiences of participants on lower incomes who are likely to be impacted by these changes more severely, but also collect views of those who on paper may not experience these severe changes but in reality may have unique views and experiences.
The second sample consideration is the size of the sample.
Again there are two questions which need to be considered:
- What fits within capacity constraints? When examining the extent to which we would like to engage with the sample, we need to understand the resource implications - for example, the time it would take for planning, research and analysis. We decided throughout the pilot year we would use 35 households (one individual in each), as this allowed us to get the balance between time pressures and the ability to explore differing perspectives. On reflection, this size and level of engagement was very resource intensive and required one researcher being actively involved for upto 5 days a month on average with another 3 or 4 researcher conducting interviews.
- When do we reach saturation? Choice of sample size can vary based on the depth and breadth of the topics to be explored. Whilst we wanted to go in-depth into household finances, we also wanted to have the ability to explore wider consumer topics such as smart device ownership and travel. Not all consumers will have experiences in these specific markets, so we chose a slightly larger sample size to provide adequate breadth, covering more niche topics. However, if research were to focus on one or two in-depth topics only, a smaller specialised sample may be preferred. The research may reach a point of ‘saturation’ sooner (the point where you will no longer find anything new).
The importance of creating an identity in a virtual world
The pandemic changed the way in which research is conducted, allowing us to make greater use of online methodologies. Whilst this provided opportunities to reach people who may not have previously been involved in research due to accessibility issues, it has also opened up a number of additional considerations. With the panel being purely online we needed to consider how best to engage participants in this long-term activity to manage the risk of dropouts.
- Setting up expectations - prior to taking part in the study, participants need to understand expectations, including: time commitment, the rationale behind the panel and engagement needed. Managing expectations early on is vital in limiting the risks of dropouts.
- Creating an identity - for many being part of an in-person panel creates a group identity; a sense of being part of something. This is something which may be difficult to achieve within an online methodology, particularly one which is primarily reliant on interviews with limited group interaction. To overcome this within the panel we provided a Welcome Pack. Within this we set out what is involved within the year, incentives (including both a monthly payment with additional payment for interview completion), any context needed, an opportunity to get to know the researchers and FAQs. The aim of the pack was to provide a resource that participants can refer to when needed. We also ensured this was provided both virtually and physically to participants in order to be inclusive.
- Inclusive and interesting methodologies - whilst the panel primarily uses in-depth interviews via video meeting software as a methodology, we also use online community software (Recollective) to conduct pre-tasks (before interviews) and any ad hoc work. Within both methods, it is important to ensure inclusion by providing opportunities for support (e.g. tech sessions as needed_ and alternatives such as phone calls rather than video. Having a variation in methods can also aid in engagement. For example, within the pre-tasks we vary the types of tasks participants are required to do (including multi-code questions, rank and sort, and open responses). Whilst this may make activities more interesting for participants you also need to ensure this is not overly complex and restricts participation.
Final thoughts
We are continuing to learn about and adapt the Consumer Insight Panel. So far the panel has allowed us to work towards our aim of inclusive research, especially exploring the views and experiences of those on low incomes. We are using the insights from the panel to feed into our advocacy work, this includes:
Sustainability - what does it actually mean to consumers?
Cost of living: an uncertain future for consumers
Christmas Shortages: Keep calm and keep it simple
The panel has also provided new opportunities to collect insight into consumers' lives longitudinally, allowing us to explore how behaviours and attitude change over time. This is our pilot year and there are still questions and considerations to be explored.
Author Bio: Paige has been a Policy Researcher at Which? for two years. She started out using primarily quantitative methodologies, but over the past couple of years has widened her research expertise to qualitative methodologies such as the Which? Consumer Insight Panel explored in this article.