Are you qual? Are you quant? Are you mixed-methods? Are you an ethnographer? A participatory researcher? Some of you may have the privilege (or arguably limitation!) of always focussing on one type of social research. Despite many of us thinking we are one particular category of social researcher, most researchers (at least in government or in private research agencies) need to be able to apply themselves across methodologies. Or as Civil Service speak would say, to ‘flex’.
When I took on my first role in government, I confess to a long sigh when allocated a role on one of the Department’s flagship surveys. My work had historically been qualitative (with specific interest in qualitative longitudinal research and researching ‘lived experience’). Accordingly, I was perhaps uninspired as well as a touch apprehensive. Quant is all about numbers yeh? And deep down, I think I had a small sense of that ‘qual snobbery’ that arguably exists in academia (or actually maybe it was borne out of the fact I scored worse in my quant module at Uni).
So what do you do when you feel a little (lot) out of your comfort zone? Well, my approach was a) don’t panic (everyone has been there, haven’t they?). Actually, I did panic a bit. Especially when people started talking as if they were SPSS and SAS technical experts. So b) was to embed myself with understanding the techniques, the IT and topic areas. No specific advice here – you may be a ‘you tube’ learner, you may learn best from others, from podcasts or from formal textbooks. Or a combination. As it was for me. Probably most useful was biting the bullet and just asking questions, even if they seemed a bit stupid (bizarrely, they never do from the other side).
But c) was and is undoubtedly the most critical. REMAIN CURIOUS. To an extent, I always think being curious actually is social research. Question (to yourself and others) why you are doing things the way they are. Be wary about metaphorically stepping over the dead bodies – the old established ways aren’t necessarily the best ones. Over the next six months, I started to question our sampling approach – and found that it had potential flaws – or could certainly be improved. I successfully argued for a stratified sampling approach. But implementing it wasn’t easy. Barriers kept popping up.
Which brings in d). Be persistent. This is not about being a methodological purist, but simply if you care about your evidence, you want it to be as accurate and robust as is possible. So those barriers, some were in-built in people’s mind sets, some were real. But I was determined to overcome them. After all, if not, how can people have true trust in what you do if you know there are in-built flaws?
In the past, I’ve occasionally been cynical about the determination or even (lack of) care that some research contractors take to underpin quality. On other occasions, I have been incredibly impressed (more of that later). The Aqua Book makes the point about QA crystal clear.
“Quality assurance considerations should be taken into account throughout the life cycle of analysis and not just at the end.”
It’s not just about verification and validation, important though they are. I eventually won my battle to improve the sampling approach. A small but important win.
Which brought me on to designing a brand-new questionnaire the following year. Questionnaire design must be one of the most important but underrated skills that exists in social research. It involves so much: not just the design of questions, respondents’ cognitive responses, stakeholder management, routing, cognitive testing. The list goes on. One of the very best training courses I ever went on was an SRA one on ‘Questionnaire Design and Testing’. I still refer back to the notes even now.
My point is not the specifics though. As I said, I thought questionnaire design would be relatively grey and dull (especially as the topic didn’t especially excite me). But I am now really passionate about holding contractors to account: making sure they avoid straight-lining, how they avoid satisficing and optimising, how to mitigate recency effects, how they consider question order effects. I could go on … and on … I’m probably now a bit of a questionnaire geek.
Occasionally, you get the pleasure of working with a contractor who has a true methodologist at the helm. Last year – through a twist of fate – I was fortunate enough for that to happen when working with Patten Smith (erstwhile chair of the SRA) on the Food and You 2 survey.
I had developed an interest in push-to-web methodology but an opportunity arose to conduct an experiment around early completion incentives. We worked together in considering the existing (limited) evidence around this issue and developed an experiment that found higher address-level pre-first reminder response rates were obtained in experimental conditions than in the control (more details here). For those interested, a paper is due to be published shortly in Social Research Practice on this very issue.
So to my total surprise, I have developed quite the interest in various aspects of quantitative research. Perhaps not necessarily the topics covered (although some were captivating). Some weren’t though; I always joked with a colleague about the mundanity (albeit high policy importance!) of some questions, particularly ‘What do you think the temperature inside your fridge should be?’ (the right answer from memory was between 0 to 5 degrees). Which triggers another point around surveys – they usually measure respondent attitudes and experience. Food and You definitely does that, but highlighted above, surveys also measure knowledge which is invaluable for organisations to understand the efficacy of their overall remit. So whether it was to understand whether people know what the temperature inside a fridge should be, whether it is how many people experience food insecurity, or whether people trust the food supply chain, I have certainly found it fascinating how complex and multi-faceted putting together a questionnaire can be.
It doesn’t mean to say I don’t still love qualitative research. Or appreciate its value. It just means I don’t sigh if I’m exposed to a new research technique or methodology. There’s always something new to learn in the world of social research – that’s its beauty. Just remain curious.
AUTHOR BIO: After working in the private sector as finance director of a medium-sized car auction in South Leeds for 17 years, Richard switched career and subsequently worked in developing outreach projects within the advice sector. After completing a Masters in Social Research methods at the University of Leeds, Richard is now a Principal Social Researcher with the Department of Work and Pensions. He lives in York, is a massive West Ham fan and still loves the Clash!