There were several factors that influenced the methodological choices:
- HSE has long running time series, stretching back to 1990s. Changes to methodology can disrupt these, so it is important to have a clear understanding of the impact of changes would be.
- HSE consists of not just lengthy interviews with household members, but also includes nurse visits to collect blood samples and other biomeasures.
- Within each household not just adults, but also children are surveyed.
- Last but not least, HSE is a random probability survey, relying on interviewers to achieve high response rates and thus reducing risks of sample biases.
On balance, it was decided not to rush an alternative mode in spring 2020, but to pause the fieldwork until restrictions allowed us to continue and instead focus our efforts on a stand-alone methodological study to investigate the feasibility of using alternative modes on HSE.
Feasibility Study
We set up a Feasibility Study to test push-to-web methodology on HSE. The key questions we were seeking answers to were:
- What proportion of households will respond and in which modes?
- What proportion of participants will agree to a nurse visit?
- What proportion of participants will agree to data linkage and recontact for future research?
- How will the estimates for key indicators be affected by mode change?
We selected a random probability sample of addresses and invited households living at these addresses to take part in an online survey. As is standard to push-to-web surveys we also sent two reminders and a paper questionnaire. The questionnaire was reduced to 20 minutes (from 50 minutes) and transformed to a mobile friendly, online survey. Cutting the questionnaire down to 20 minutes meant difficult decisions needed to be made on which topics to include and how to adjust the existing questions.
So what did we find out?
As expected, the response was a lot lower than is usually achieved in the face-to-face survey - around 25% compared to around 60% in previous years. Encouragingly, most responses were given online (70%) rather than by post. The number of child interviews we achieved was a lot lower than we expected, probably because we used a two-stage approach with adults interviewed first and invitations for the child interview sent out later. In the future, it is worth considering whether adult and child interviews can be carried out during the same stage.
There are some groups who are less likely to respond in most surveys: men and younger people, for instance. However, in addition to this, people from more deprived areas and renters were also less likely to respond, which is something we do not see in the face-to-face HSE survey. This shows the important role interviewers play in convincing people from these groups to take part.
The HSE asks for four types of consent from participants:
- To be contacted for a nurse visit;
- To be contacted for follow-up research;
- For their data to be linked to health records; and
- For their data to be linked to mortality data.
The consent rates to all four were lower in the Feasibility Study compared to the face-to-face HSE survey. The decline was highest for nurse visits, with only about half of Feasibility Study participants (52%) agreeing, compared to 85% in the face-to-face survey. This is important, as it significantly reduces the number of participants from whom we could potentially collect biomeasures.
We also looked at some key estimates from the Feasibility Study and compared them to the most recent face-to-face HSE data. We found that despite weighting the data, the Feasibility Study participants were less likely to report that they had a longstanding illness and to be current smokers, but more likely to report that they were not physically active. This suggests that the Feasibility Study underrepresented some groups even after weighting was applied.
Informed by these findings, NHS Digital concluded that the issues identified by the Feasibility Study were too substantial to move HSE immediately to push-to-web design. However, it has been a great learning experience and we continue to explore ways to improve our approach.
Main Health Survey for England survey
The main HSE survey was only paused temporarily during the pandemic, and is now delivered both through face-to-face and telephone modes. We adapted the HSE interview for telephone administration (making it shorter, finding a way to give participants showcards, changing interviewer instructions) and used this mode throughout 2021. During the third national lockdown in January to March 2021, we invited people to opt-in to the telephone survey, but since April 2021 the interviewers have been knocking on people’s doors, persuading them to participate in the survey. Interestingly, we saw a slightly better response to the telephone opt-in compared to the push-to-web Feasibility Study!
We also investigated the option of offering video interviewing on the HSE as an alternative to telephone. We found that the interview worked well over video and both participants and interviewers were happy. The proportion of people opting for video interview was small but steady during the opt-in fieldwork (8% of all interviews during this period were conducting by video). However, as the country emerged from lockdown and interviewers resumed door knocking, the interest in the video option waned. It seems that most participants are happy doing the interview over the phone, so we have stopped offering it on HSE.
Given the uncertainty of how the pandemic was going to develop, the HSE 2022 launched as a knock to nudge survey in January, with interviewers knocking on participants’ doors to arrange a phone interview. However, as the restrictions have been eased, the interviewers have been able to carry out interviews in participants homes from February.
Visit NatCen’s Centre for Social Survey Transformation to find out more about how we are supporting transformation in social survey research.
Author bio: Mari Toomse-Smith is the Director of Health and Biomedical Surveys at NatCen. She leads the team of 20 researchers and oversees NatCen’s health surveys portfolio, providing strategic direction and quality assurance.
Discover more in the Explore the Data series:
Explore the data: the British National Surveys of Sexual Attitudes and Lifestyles (Natsal)
Explore the Data: The 1970 British Cohort Study (BCS70)