Methods Matter Podcast – Qualitative Secondary Analysis

Leah Fullegar:

Hello, and welcome to the last in first series of the Methods Matter podcast from Dementia Researcher and the National Centre for Research Methods, the show that throws a light on different research methods, helping you to decide what suits your research design. In this series, we’ve been looking at five different research methods with an expert from the field and a dementia researcher that has put the method into practice.

Leah Fullegar:

My name, for the fifth and final time, is Leah Fullegar. I’m a PhD student at the University of Southampton, and I research dementia care and faecal incontinence. This podcast came about when I got to the research method section of my PhD writeup and realized I can’t cope with this, this is too hard. Together we’re going to work out where I went wrong, learning a whole load of things I wish I had known three years ago. Today, we’re picking over the bones of our data, recycling, reusing, and realizing that there is life in the old dog yet and exploring qualitative secondary analysis, which I really struggled to say, so I’m going to say QSA from now on.

Leah Fullegar:

Joining me on the quest are two truly inspirational guests. In our expert corner, we are once again joined by the fountain of knowledge that is Dr. Kahryn Hughes from the University of Leeds. And over in research around rehab, the incredibly hardworking brilliant Dr. Anna Volkmer. Kahryn is director of the Timescapes Archive, Editor in Chief of Sociological Research Online, convener of the MA Qualitative Research Methods, and a Senior Fellow for the NCRM. Anna is a Senior Research Fellow and speech and language therapist at University College London. She is particularly interested in language led dementia and primary progressive aphasia and developing interventions to remove communication barriers… and she’s a bit of a huge superhuman. I’m so looking forward to the show. Hi, Kahryn. How are you? It’s great to have you back.

Dr Kahryn Hughes:

Hi, Leah. Thank you very much for having me back. I’ve been really enjoying these podcasts, looking forward to this one.

Leah Fullegar:

Fantastic, thank you. For anyone who has only just discovered the Dementia Researcher podcast, Anna is a regular contributor as a host and a guest. Hi Anna, how are you?

Dr Anna Volkmer:

I’m good, thank you. I’m looking forward to being on the other side. I’ve been hosting podcasts recently, so it’s lovely to be on the other side.

Leah Fullegar:

Does it feel a bit more relaxed?

Dr Anna Volkmer:

It does actually, yeah.

Leah Fullegar:

I loved your recent vlog on open access publishing, and I spotted that you’re guest editor for the special collection coming out next year on creative approaches to positive dementia research. Do you want to get a plug in for that?

Dr Anna Volkmer:

Oh, yes please. As you said, I’m co-guest editor on a special collection for the International Journal of Qualitative Methods on creative approaches in qualitative research. We’re looking for articles, I guess, which take a creative approach to redesigning qualitative research methods and research practices so that they’re more inclusive. And so they kind of fit the competencies of the people involved, so people with dementia who might be involved in the studies. We want the collection to reflect work where the voices of people living with dementia are actually present, or guide the research. The plug is, abstracts are due by 31st of January, so please do submit. You can find the link on the Demiqual Twitter handle, which is @demiqual… D-E-M-I-Q-U-A-L, Demiqual.

Leah Fullegar:

Fantastic. I hope everyone listening wrote that down because I’ll probably forget it. What do I know? We begin each podcast with me giving a summary of what I understand about the method we’re discussing, which of course today is qualitative secondary analysis. Have to pause before I say that.

Leah Fullegar:

This is my chance to shine because this is one I actually know something about because it does mostly what it says on the tin. I would say secondary analysis is about using existing data collected for purposes of a prior study in order to pursue research that is distinct from the original research. It’s looking for different aims and answering different questions. It may be a new research question or an alternative perspective on the original question. I’m now going to go to Kahryn and see what was wrong with that definition because I’m sure there was quite a bit.

Dr Kahryn Hughes:

There was nothing wrong with that definition. It was really, really good. What I would say is that in actual fact, qualitative secondary analysis also includes extending existing research. It can be described as a continuous form of QSA, and that’s language used by Professor Sarah Irwin at the University of Leeds, but also by Dr. Anna Tarrant, who has used a form of continuous QSA in her work. Qualitative secondary analysis can also be of any sort of data, but it’s how you treat those data and how you render them as qualitative. That’s one of the tasks, if you like, of qualitative secondary analysis.

Leah Fullegar:

Does it count as secondary analysis if halfway through your data collecting you realize that there’s another question that you could answer that’s related, but not your original one?

Dr Kahryn Hughes:

Is this during your own field work?

Leah Fullegar:

Yeah. Yeah.

Dr Kahryn Hughes:

I wouldn’t really use the language of QSA there. I’d just say you’re tacking backwards and forwards really, and that your research has actually been quite successful. You’re finding out new things and it’s producing new questions that you are then going on to interrogate. But I think that’s a really, really interesting question in that it’s asking about timeframes, how far back. When we use the language of qualitative secondary analysis, we’re always sort of invoking this idea of going back and returning to, and your question suggests that it can be very much in the moment. I think that that’s right. Timeframes can be infinitely variable in QSA.

Leah Fullegar:

Would that mean that the definition of QSA would be that when data collection is finished, when does it become QSA rather than just tacking backwards and forward in your data collection… or is that a loaded question?

Dr Kahryn Hughes:

No, I think that that’s an absolutely fascinating question. I think that that is almost where QSA was born. Because in a previous podcast I was involved in on qualitative longitudinal research, the actual methodology of QL involves going back… going back and revisiting data and thinking through from the vantage of the present, whenever that was or is. What may have changed, what continuities there are, so on and so forth. And we ourselves and our perspectives may have changed in making sense of those data.

Dr Kahryn Hughes:

I think that when we would describe when we’re talking about QL, we were saying it’s a methodology that’s fundamentally attuned to questions of time. I would say that in part, so too is QSA. It needs to engage with questions of what I would describe as temporal remove. How far in time are we, what sort of distance, how far that might be a limitation. I think that the emphasis has sometimes been too far in that direction. But also, how we might harness the affordances of temporal remove. How much time has passed allows us new sorts of perspective on ways of thinking at a given time.

Dr Kahryn Hughes:

When we talk about continuous QSA, we’re talking about contiguous research, research that’s perhaps going on at exactly the same time as the original study. Another researcher may be going in and repurposing those data for new sorts of questions or repurposing those data as evidence for a new type of study. That temporal remove can be, again, infinitely valuable.

Leah Fullegar:

That seems to be one of the common answers in qualitative research is that there’s infinite answers… try and decide on one that fits your particular research and your particular study.

Dr Kahryn Hughes:

Well, absolutely. There’s that pragmatic element is what suits me and how might that suit the aims and intentions of my study. But also, that’s one of our analytical tasks then is how we theorize those data as a particular form of evidence, how we make a case that those data are relevant to speak to the sorts of questions that we have. That temporal remove is absolutely one of the dimension that we have to engage with theoretically.

Dr Kahryn Hughes:

The point is, you can’t just stroll into somebody else’s data set and use it uncritically. That doesn’t work at all. It’s this, what keeps coming back is infinitely variable is as if it’s how long a piece of string, it could be anything, we don’t need to worry about it. I don’t think that’s the case. I think that infinitely variable means therefore it’s a job of work for us to do analytically in making sense of the value of those data for our own research and recognizing how we are situating those data as a particular form of evidence that can speak to the questions that we’re bringing to you.

Leah Fullegar:

I will just say, because this is where you’re a part of the Timescapes Archive. QSA must be incredibly relevant to the Timescapes Archive. Is it something that’s used a lot?

Dr Kahryn Hughes:

Well, the Timescapes… the original ESRC and Timescapes program had three strands. One was seven empirical studies that were all QL and they are heritage studies. The second was the development of the Timescapes Archive for the purposes of reuse, and it sits as a satellite archive to UKDA because it’s developed in collaboration with them. And the third strand was, the possibilities for qualitative secondary analysis. That was led by Professor Sarah Irwin. She was absolutely instrumental in developing language of QSA. And it was a task that everybody involved in Timescapes was involved in, in one form or another.

Leah Fullegar:

So you really are the perfect person to be talking to right now.

Dr Kahryn Hughes:

One of them. As we’ve said in previous podcasts, these are collective endeavours. We’re not this solitary candle shining light on a particular methodology. We’re always in dialogue, we’re working always. If we’re not working on other people’s work, then we’re doing something wrong. It’s not scholarly. Yeah.

Leah Fullegar:

On that note, Anna, let’s bring this back to dementia. Can you tell us how you’ve used secondary analysis in your own research?

Dr Anna Volkmer:

Yeah, absolutely. When I was first asked to do this podcast, my initial reaction was, “Yes. Yes, of course.” And then my second reaction was, “Oh, do I actually do secondary analysis?” But I was saying to Kahryn and Leah, I think actually in my field, we do it all the time. As an example, in my PhD, I did a pilot feasibility study. I developed an intervention for people with dementia. Part of the study was to trial it, so to look at the feasibility of delivering the intervention and we collected heaps of different outcome measures.

Dr Anna Volkmer:

We delivered it within the NHS and we asked the local speech and language therapists to collect as one of the outcome measures, video recordings of the people with dementia, they have this language led dementia or primary progressive aphasia. And we asked their speech therapists to record each of these people with their partner. We had couples, and they recorded 10 minutes of their conversation four times. We had four 10-minute samples pre-having the therapy or not having therapy… that was a randomized trial… and 10 minutes of them having a conversation four times after having therapy. In other words, we had 80 minutes of video data. In the actual study, we collected that data from 18 participants. And in the original study, I was particularly interested in the viability of… I mean, the feasibility of collecting the data within that pilot feasibility study, but also kind of looking at the behaviour. We actually counted behaviours as kind of a more… we coded and counted behaviours.

Dr Anna Volkmer:

However, since the study’s finished, we’ve gone back to this video data, this huge archive. I mean, that’s a lot of video. That’s eight times 10 minutes for 18 participants. We’re not using all of it, but we’re looking at some of the data. And we’re using conversation analysis methods because what we… in this field, what we don’t have heaps of information on is how conversations work. There is some research data on that, but there’s not heaps. This is probably one of the biggest video or collections of videos of conversations between people and their day to day partners, natural conversation samples. Conversation analysis is based on natural conversation samples. It’s a great resource to use, and to examine the patterns and organization of conversation between these people with these different types of primary progressive aphasia and their day to day partners. Does that make sense?

Leah Fullegar:

That’s fascinating, to be honest. I was going to say it’s lucky, but it’s not lucky because you did the first study. But it’s amazing that you have this huge amount of video data that is quite difficult to get in other studies if you didn’t have that. If it wasn’t already your data, is it quite difficult to access data to analyse for you to use for QSA?

Dr Anna Volkmer:

That’s a really interesting question. I have to say within my disciple, so as a speech and language researcher, there are archives as Kahryn described and they’ve been set up within the discipline, within the profession, so to speak. There’s a number of archives where people submit audio or video data of conversation or natural interaction. There’s a lot of that, or even discourse. There’s lots of archives. There’s people writing descriptions of holidays or descriptions of pictures, common pictures that are used in common tests or conversation data. It’s been done with children. More broadly across this type of speech and language research, there’s very well-known archives of child development data, of data of people post-stroke. We call that post-stroke aphasia, post-stroke language impairment.

Dr Anna Volkmer:

What there hasn’t been however, is much data collected in my field on people with dementia. We have added the archive we create we’re trying to start this… well, complement the stroke archive, really. And because many of the speech and language researchers who’ve studied stroke aphasia have also been exploring language and dementia, some of the research is the same people. We’re adding our archive of data to that archive. But actually, I think perhaps having spoken to my colleagues and supervisors and mentors, it’s a culture within our discipline of sharing data. That’s what helps. And it’s a great source type of data to share in that way. I don’t know, Kahryn, whether it’s so common across other disciplines to be so collaborative.

Dr Kahryn Hughes:

That’s a great question, Anna. This is, I think, one of those characterized QSA in the social sciences. It has been quite a contested field. For example, in normal history if you were in normal history, it wouldn’t be. People are quite used to reusing sample sources and the QSA of those sources. Obviously, it’s par for the course. For example, in sociology, there has been a long tradition of ethical development around how we treat research. The research participants, how we treat their data set and so forth, and what our relationship is to them. I think that these have been incredibly important. I think that they’ve influenced society more generally around the ethical treatment of people and with data.

Dr Kahryn Hughes:

However, historically, the concerns were very much about for example, the systematic misrepresentation of participants. If researchers changed those participants or perhaps hostile even to the original aims and intentions of the originating researchers in the study, they may systematically misrepresent. Though also, there was quite a long period where people, social scientists, describe participants as vulnerable. That built out of, I think, attention to the asymmetrical power relations in research and research relationships.

Dr Kahryn Hughes:

The ethical thing was historically and for many decades, was to collect data, analyse this data, and then destroy them because what we did was then protect those participants. They couldn’t be tracked down, their data couldn’t be misused, but you protected their anonymity and confidentiality and so forth. That seemed to be the ethical thing to do.

Dr Kahryn Hughes:

And then in around about 2004, Jennifer Mason did a scoping review. I think she was commissioned by the ESRC to do that. It might have been a little bit later, but it doesn’t matter, a couple of years here or there. One of the things that she pointed out was that qualitative data were really expensive resources to produce. And there was also… I mean, this is something that I would add. I’m not sure if Jennifer said this, she would have done. But there’s this ethical attention that we need to pay to the contributions of our participants. They’ve given their time, they’ve given their experiences, they’ve really contributed in a number of ways. We know that because there’s lots of work on the emotional burden of research, of both researchers and research participants.

Dr Kahryn Hughes:

There was a bit of a change in attitudes. The ESRC invested in a couple of experimental or exploratory studies and groups of studies looking at the possibilities of qualitative secondary analysis. On the basis of the findings of those, it funded Timescapes, and that was why the third strand of Timescapes was qualitative secondary analysis. If we build the archive, will people come? That sort of thing. Yes, they do.

Dr Kahryn Hughes:

Subsequently, there’s been a tremendous body of work not only within sociology, but across the social sciences and disciplines more generally that I think is very much associated with other broader changes such as identification of society and massive drive to collect and analyse digital data, so on and so forth. And the development of archives and repositories in the technologies of both storing, curating, but also making available large bodies of data. There has been multiple developments in multiple directions that have created, I think, a more engaged community in social sciences who are beginning to recognize the value of reusing data.

Dr Kahryn Hughes:

One of the things that myself and my colleague, Dr. Anna Tarrant, has written about has been the development of what we describe as a temporal ethical sensibility. Because when we think about qualitative secondary analysis as tendencies are said to look back. But I think that also, it encourages us to look forward. Something that Andrew Clark talked about in the QL podcast.

Dr Kahryn Hughes:

But here, if what we recognize is that particular individuals and groups in society struggle to participate digitally and struggle to participate in social research, that when we do capture some data, we do capture encounters and interactions with these people, what we’re facilitating is an opportunity for them to tell their story and to speak to particular political agendas to see what it is. When we’ve spoken to those people, they often describe their lives as hugely constrained in terms of being able to speak to political agendas. They were often overlooked, they were unheard, or they’re simply not engaged and involved. We can see them as marginalized in two ways, both in terms of having the capacity to speak to political agendas and being heard, but also marginalized digitally. They’re not being captured in the digital transaction data. So they’re not hearing them in those big data.

Dr Kahryn Hughes:

For me, there is this responsibility to think through how we preserve those accounts for future researchers. Those aspects of social dynamics, social relations, social processes, are part of future historical records. That avoids what I would describe as a double silencing, which is not only they’re marginalized and silenced in their everyday lives, but also when we erase their data, we subsequently erase them from their social historical record.

Leah Fullegar:

That was so helpful, and such a poignant description. Actually, it links back to the example I gave where when I took the project to NHS Ethics, to the REC Committee, they were hugely concerned about vulnerability of people with dementia. My participants, the advisory group, the PPI reference group that I work with, strongly felt that people with dementia… that the people in our study would most likely have capacity.

Leah Fullegar:

And even if they maybe were at risk of lacking capacity, it could be that the consent process could be appropriately undertaken to determine whether that person wanted to be recorded and that that person and their family member could make the decision about whether they wanted their data to be kept. And we incorporated it into our consent forms in an accessible aphasia-friendly way. When I asked them whether we should incorporate it, the people in my advisory committee, my PPI reference group, were horrified by the idea that their investment in data… when they were living with a progressive disease, a disease that progresses actually quite quickly in terms of language and communication, and might not have any longevity and that it might not benefit more than one researcher. They were absolutely horrified that the REC could take that decision away from them. It was really fascinating.

Leah Fullegar:

And then in the REC, they, I guess, tried to marginalize this community by inferring that they didn’t either have the capacity to make that decision and that perhaps the consent form we developed in an accessible way, co-developed, didn’t need to be accessible and didn’t need to be co-developed. Of course, luckily for us, they then acknowledged that that was the right process and we were able to collect this data. But everything you said resonates with that experience that me and my team had. So thank you for naming some of those things, it’s really helpful.

Dr Kahryn Hughes:

I think it’s a critically important example. The thing that I would say when we’re talking about a temporal ethical sensibility, that’s Anna and I talking about this, is that if you properly curate data and store data as with in your profession, Anna, that what you build in is the potential for changing ethical sensibilities and changing ethical protections. Rather than becoming very laissez-faire and say if only one can use these data… whatever. We know that qualitative data have very particular qualities which do mean that they are very sensitive and they do require ethical management and handling.

Dr Kahryn Hughes:

But when you archive those data, what you do is you make visible who is using them, you facilitate conversation and dialogue. For example, with the originating researchers and with the researchers that will be using those data, or if those time periods are much longer with those that are curating those data or responsible for managing and protecting those data, you’re able to facilitate a dialogue and make visible the ethics of how those data are being reused.

Dr Kahryn Hughes:

One of the problems I think is that when we think about ethics, we only ever think prospectively. It’s as if somehow when we do, ethics when we go through REC and so and so forth, that everyone can feel hugely confident that we’ve ticked all the boxes and it’s going to be ethical research. And yet, any qualitative researcher who has been involved in qualitative research knows that in research there’s always this uh-oh moment where something is not right. You’re going to have to bring in emergency measures, you’re going to have to do this, that, and the other.

Dr Kahryn Hughes:

There’s been a lot of writing on that emergent negotiated ethics in qualitative research. So we have to think about prospective ethics. But also situated ethics, ethics in the moment. When we begin to think properly about QSA, about archiving things like that, we can think about those retrospective ethics. We can think about those ethics that are going to be in the future and employed by researchers considering how they may be using those data. We build in the prospect and the potential for new and emergent ethics in the discipline and in the field and in the management and treatment of archived data, and in particular studies.

Dr Kahryn Hughes:

There’s this other aspect as well which is that when we think about the re-use of data, it all seems quite hidden away. And yet, what we have are people reusing data in order to write papers or to develop interventions or to write reports and so on and so forth. When we begin to think temporally in those ways, we can also observe the opportunities for scrutiny that are built into say, publication processes. For example, through peer review and so on and so forth. These opportunities for dialogue and scrutiny and consideration and negotiation and so on and so forth, are built in to thinking temporally about lifetimes of data, how these data are going to be managed in their own lifetime, as it were.

Dr Kahryn Hughes:

I hope that helps. I hope that ticks a few boxes, particularly with the people who are thinking of going through ethics, that that is something that they can say if my data are going into an archive, this isn’t making my participants more vulnerable, this is in actual fact avoiding a double silencing and it is building the potential for increased ethical scrutiny and management.

Leah Fullegar:

That was a fantastic conversation to listen to. What we’ll do now is we’ll get into some detail and provide some top tips for anyone who is new to using the method. In this segment, I’m going to ask some quick straightforward questions to both guests on how to put this method into practice. Kahryn, the first ones are for you. Why do secondary analysis?

Dr Kahryn Hughes:

I think, again, we’re going back to the historical thing. One of the things that was really clear during the COVID pandemic which is still continuing, it is not over, is that the disproportionate impacts of COVID were falling onto those individuals and groups that systematically experience disproportionate impacts. What we find is that it is very gendered, it is very racialized, and so on and so forth.

Dr Kahryn Hughes:

One of the things about QSA, what that allows us to do, is to bring historical analyses to bear. Where historical might be only over the last 5 to 10 years… it might be over 20 years or maybe even 100 years or whatever… but particularly in the context of COVID, it was very clear that certain forms of disadvantaged meant that certain individuals and groups were unable to… were constrained in how they managed and negotiated those impacts of COVID. And rather than being in a position where we say, “Oh, why would that be?”… QSA analysis to engage with not only the data, but also the writing and so and so forth… I see this as all part of the same thing, other forms of evidence… to make sense of how and why the impacts of COVID were disproportionate for particular individuals and groups. Does that make sense?

Dr Kahryn Hughes:

This can’t be quick, I’m so sorry. I was doing two studies at the same time. I was looking at midlife grandparents in low income context at the same time as looking at the impacts of internet gambling on the family. I had been doing all the analyses for the internet gambling study whilst setting up the midlife grandparents study. Once that was under Timescapes, the midlife grandparents study is under Timescapes. And then I went back to the internet gambling data. All of these grandparents fell out of the cupboard, there were grandparents everywhere, and I had not seen them. I had not seen. I had been so preoccupied with questions of addiction, temporal analyses of addiction and so and so forth, which built out from previous work that I had done. Myself, my own perception, my own awareness of what the questions might even be and how I might bring them to sets of data, changed simply because I had been involved in another study.

Dr Kahryn Hughes:

One of the things for me is that qualitative data can never be analytically exhausted, ever. I’ll align myself with if I might, with Jennifer Mason, a big giant in methodology, and I’m being cheeky here. I think that these are hugely valuable resources. They’re expensive to produce, but also they’re enormously valuable documents of life.

Leah Fullegar:

I think you did very well to get five years of work into that short amount of time, so well done.

Dr Kahryn Hughes:

Thank you.

Leah Fullegar:

How do you prepare your data for secondary analysis? I understand that’s probably another how long’s a piece of string question.

Dr Kahryn Hughes:

Well, if you’re presuming that those are your own data. And often in qualitative secondary analysis, we’re working in other data sets or across other data sets or potentially across different data repositories. For me, there were two questions here. One is, how might you prepare data for archiving or for qualitative secondary analysis? That’s far too detailed to go into. Professor Bren Neale has written a very lengthy document that is on the Timescapes Archive that I would recommend any researcher go and read if you are conducting qualitative research, because it tells you how to keep and organize and store your data. It means that whenever you go back to your data, you know exactly where you were in your data set. It’s absolutely fantastic. Those guidelines were developed in dialogue with UKDA, so it’s gold standard data preparation advice.

Dr Kahryn Hughes:

The second aspect is about how you might sample your data, and I think that that’s something very, very different and it’s hugely challenging. One of the things that I think the common misconception is that QSA is easier than field work because the data are already there and you can just use them. No, no you can’t. There’s a lot of work you have to do in order to, as I said, make sense of how those data in particular form evidence… how you might even sample, how you might even control the amount of data that’s available to you.

Leah Fullegar:

To prepare for secondary data analysis, you need to do a hell of a lot of reading.

Dr Kahryn Hughes:

You have to have done the same amount of work that you would do for any sort of primary field work. You have to be clear. Why am I doing this research? In primary field work, we say, “Well, why would I speak to those people? How can they inform the questions that I have?” As Anna says, you’ve developed your questions in dialogue with those people. In qualitative secondary analysis, that is then your challenge. How and why? What sort of data am I using, and why might that be? How and why do I think that that can inform on the sorts of questions that I’m asking?

Dr Kahryn Hughes:

The point that I would make is that there was more data than we can as a species, that we can ever, ever possibly analyze. We can never exhaust those qualitative data. There’s too much. Your challenge is not so much finding the data as in managing and controlling the data that you’re going to be using.

Leah Fullegar:

Last question for you, Kahryn. I know we’ve already talked a bit about the ethical considerations of QSA, but are there any other particular main ethical issues with QSA?

Dr Kahryn Hughes:

There are, and they’re very complicated. For example, there was the one around anonymization, and this is when we think about preparing our data. Niamh Moore has said… she’s written about this. Niamh Moore has written some seminal papers on qualitative secondary analysis, by the way, in a special issue that was headed by Jennifer Mason in 2007 in sociological research online. But Niamh has written anonymization. One of the things that she says is that if you’re anonymized, you strip meaning out of the data. I think researchers who were reusing data have those challenges around how we might anonymize data and yet keep them meaningful. Again, this is retrospective ethics. I think that’s easy and manageable, you just use the same techniques as you would.

Dr Kahryn Hughes:

There’s the one around systematic misrepresentation. How do we know? How is our interpretation of these data one that can be sustained? I think that that’s not only in intellectual… well, if I can call it intellectual endeavours are ethical. But that is where we made foreground in particular sorts of ethical questions there.

Dr Kahryn Hughes:

There’s another one which is really unusual which is the professional discreditation of others. It’s very easy for people to walk into a data set and have a look at how somebody else has conducted an interview, and think that they’ve done bad. “That’s a bad interview.” Or the language that has been used at a given historical moment becomes quite pejorative and problematic, say a 10 years’ distance or a 20 years’ distance. Certainly, how we might speak about ethnicity, race, gender, class, all of these sorts of things. This is a constantly changing field. There is a responsibility to the originating researchers. The confidentiality and anonymity is quite hard when you are citing a data set and naming the people who conducted the research. There are those sorts of quite unusual ethical concerns that we may have that are very particular to qualitative secondary analysis.

Dr Kahryn Hughes:

And then, finally, there’s the question of consent. The consent is something… Bren Neale talks about the shift from informed consent to enduring consent, and this might be helpful for all researchers who are thinking about gathering research data and that they are then going to archive. Enduring consent is where you assure people that those data are going to be treated ethically, so those quite distinctive ethical aspects of QSA.

Leah Fullegar:

Anna, your turn. Are you ready?

Dr Anna Volkmer:

Yeah. I’m going to try and stay short and sharp. I’m going to talk and talk, I know I am.

Leah Fullegar:

How do you handle missing data?

Dr Anna Volkmer:

I think my example is perhaps a little bit bias because it was my own data. Within our study, we were able to go back to the source if we thought there was some data missing, go back to the original documents and the original data sets, and even some of the hard copies because they were there that we had to store within the constraints of our ethics. So we did have that privilege. The other privilege we had as well I mentioned in the original project, we collected eight recordings from every single participant, the absolute wealth of data there.

Dr Anna Volkmer:

In methods of conversation analysis, you might only select a small proportion of the conversation to analyse. What we’ve got is a plethora of information from every participant. If one video was somehow compromised or was missing… we’ve had examples not within our study… no, within our study, where one of the video… a couple of video recordings were made… because the video recordings in our study were made by the person in their own home actually. We gave them equipment, and they video recorded themselves. And of course, people video record the table instead of themselves, they video record the ceiling, the dogs come in, they actually don’t get any audio. We could make up for that by taking an alternative sample.

Dr Anna Volkmer:

We have had started working with somebody outside of our own research group who’s starting to look at our data. And actually, similarly, what they do is they come back to us with any questions. I believe in our speech and language research. It’s quite a small community, but they always come back to us nevertheless as the… I was talking to my mentor about our data and we were talking about us being kind of the guardians of the data. So they were coming back to us as the guardian of the data, I guess.

Leah Fullegar:

How do you overcome biases in the original data collection?

Dr Anna Volkmer:

Yeah, that’s a really good question. When I was looking through the script for today and some of the questions, I was thinking about how I was going to answer you because our data I mentioned just now is filmed in the person’s home by the people themselves. The idea is to reduce bias in our data set, because it’s natural conversation samples. So then with the natural conversation sample for conversation analysis, that’s the aim, you reduce the bias. You take a longer sample. We were doing that anyway, we were taking really long samples. In conversation analysis, you try and make it as natural as possible, and you often use the last five minutes of the data set because you’ve audio or video recorded some of these commonly understood within that field that the last five… if you take a 10 or 15 minute sample, the last five minutes are the more natural. We happen to be doing that anyway.

Dr Anna Volkmer:

But there are other biases. For example, when you are recording somebody in any way, audio or video recording, that people create a bias where they perform for the video. They’re the kind of performance bias, if you will. Again, we try to record multiple samples.

Dr Anna Volkmer:

The other thing we were talking about recently, I think it’s the Hawthorne Effect. This is the idea that if somebody is observed, they will perform differently. That’s based on some research. I was talking about it yesterday with a colleague. It’s based on the idea that a piece of research that I think was done in a car factory. They were saying that if they watched the workers, they performed better just because they were watched. But actually, there’s been some recent critique of that work that suggested that’s not actually the case.

Dr Anna Volkmer:

And so, I think when we’re talking about that kind of bias, how you are video recorded or whether you change your behaviour for audio or video record… yeah, sure. There may be some change, but actually, there’s also perhaps the degree of change isn’t what we think it is. Kahryn’s going to add to that. I think Kahryn will add to that in a minute. But I guess the key thing to say here is that I think that even when you are recording someone, you’re recording their conversation style, their conversational… we all have a variety of conversation styles. As a clinician therapist, I’m constantly talking to people about that.

Dr Anna Volkmer:

I’m not going therapise you and talk in one way. We have multiple identities. No matter when we’re video or audio recording, I believe that we are recording an aspect of a person. I think by recording people without the researcher present, that’s been quite a helpful way of reducing the bias. I’ve worked with data sets where they’ve had the researcher present in the data set, and that’s often more difficult to manage, I found, much more difficult.

Dr Kahryn Hughes:

I will just come in. Music to my ears, Anna. I absolutely agree with you. This is a form of performance. I’m trying to communicate to you in the clearest way that I can what my thoughts are for example, about QSA. It’s a very particular and focused sort of conversation and dialogue. It will be if I went to my GP, it would be if I spoke to the person in the Tesco’s checkout. There’s always some aspect, a formative aspect, of what it is that we do with and for other people in our families and so and so forth. These are multiple. There are just so many. I absolutely agree with you that that does not then detract from what it is the efforts that those people are trying to make. They’re trying to talk. They’re trying to explain something perhaps to each other. However performative it is in one sense or another, there is always going to be for me, that element.

Dr Kahryn Hughes:

This is a human communication. We do mediate and navigate how we might speak to one person as opposed to another. For me, that value is not taken away simply because… now we’re going to talk with you on the film, and what we’re going to talk about is… let’s talk about this, a little bit chatty. That tells us loads, it tells us oodles. I don’t think that it tracks. Deciding with bias is like… bias, okay… orientation. How are people focused in these particular ways? How might that change over the course of those eight videos? How does it change from the first five minutes to the last five minutes, and so on and so forth? But what consistence is in terms of the aphasia that we’re able to observe even when people are trying quite hard to speak to camera, and so on and so forth.

Dr Anna Volkmer:

Even when there’s a researcher involved, I’ve been using somebody else’s data that actually worked. We’re doing mixed research in our secondary analysis. But it’s got the person collecting the data. They’re asking them to recount a holiday. What’s really interesting is in some of these samples, the researcher’s saying, “Mm-hmm, yeah. Uh-huh. Right, uh-huh, and? Right, yeah.” And in others, the researcher is them. One might suggest that that’s biasing the sample. But actually, one might also reflect on the fact that the researcher is using minimal terms to support the person’s communication because they already have a communication difficulty to demonstrate the competence of the person to get the more or better sample or bigger sample.

Dr Anna Volkmer:

We might actually say that what the researcher is doing is part of the person’s communication as well. It’s really interesting and really fascinating. And perhaps, even pre-communication disorder, pre-dementia diagnosis, people have different styles of conversation anyway. You know and I know that one… we have friends who are very quiet and we have friends who are very chatty, and we have different strategies to get the same information out of them.

Dr Kahryn Hughes:

I would jump in here and say that what people can’t see is that we’re all nodding at each other. Verbal cues for people is absolutely part of everyday life. We say, “Uh-huh, yeah. Oh, yeah.” Or we say, “Uh-huh.” We use verbal cues, it’s part of how we communicate. And we use non-verbal ones as well, which is what we’ve been doing here. Therefore, it’s part of that dynamic in that way. It can be treated as a form of data that can inform on the type of dynamic that was and wasn’t possible in those conversations with those people. Yeah.

Leah Fullegar:

The final question I’ll ask you is, will this method save you time?

Dr Anna Volkmer:

My experience is, it took longer up front when I was planning to keep my data for long… invest in that idea of other people and myself being able to do qualitative secondary analysis. I found for my own analysis… I have found it feels quicker on the back leg, even though I’ve had to do planning, I haven’t… it felt quicker because… I think because I haven’t had to jump through some other hoops that I found quite difficult to jump through in the first place. I think I would call it an investment up front for a long-term gain.

Leah Fullegar:

Thank you very much. It’s time for a recap. There’s a hell of a lot to recap. We’ve been talking about so much. Healthcare research requires significant time and resources. The wealth of data and information in qualitative research is just so worth revisiting and going over and over and over, because we can never analyze it all. We can never fully get to grips with all of that data that exists in the world. Everyone who’s listening to this needs to go out and do some qualitative secondary analysis. And one day, I will say it correctly.

Leah Fullegar:

It’s time for the final part of the show where we discuss the pitfalls so we can help you jump over them. Anna, can you tell us if you came across any challenges in your research, and what you might do differently if you were to do it again?

Dr Anna Volkmer:

Kahryn actually touched on one which is data storage and transfer. I thought about data storage, but I hadn’t thought about the devices. In qualitative research, you often generate large data sets, and I was collecting videos and audio recordings which are huge. I would say you need to think not only about how you’re going to store it, but what on, and make sure you’ve got plentiful tech, basically, hard drives that are able to cope with that.

Leah Fullegar:

A big filing cabinet full of massive hard drives full of data. Kahryn, can you think of any other common pitfalls of QSA and how to avoid them?

Dr Kahryn Hughes:

A lot of what we’ve been talking about has been what Anna and I were talking about depth-to-breadth qualitative secondary analysis, which is working with small qualitative samples and building up and building to bigger sorts of analyses.

Dr Kahryn Hughes:

There is some other work that was led by Rosalind Edwards, Lynn Jamieson, Susie Weller, and Emma Davidson, funded through the NCRM. They developed methods of Big Qual, which is where you can run… a program for example, like Leximancer or so forth, and amalgamated qualitative data and identifying new things and so forth. But even when you are using software or whatever, qualitative data are hugely time consuming to analyze. At some point, I can remember when they first reorganized the Timescapes Archive. I had a sandpit with the library team who had done all the tech work and everything. We organized how we might browse and search in the archive and things like this. We’re super excited. I tried to dive into the data. And basically, what you do is you open up an interview, for example, something else. You need to make sense of it.

Dr Kahryn Hughes:

You need to understand where this interview might sit in the context of a broader data set who this person is. What were the purposes of the original research were, how old are they? What was their geographical… and so on and so forth. There were those sorts of things about contextualizing or re-contextualizing the data, which is hugely time consuming. That means you have to do a lot of reading around those data. You need to understand what they are and where they’re from and so on and so forth. And reading a transcript and analyzing a transcript is just as long and arduous to QSA as it would be in any sort of primary field work. That’s one of the potential challenges. I’ve already mentioned another which is sampling, which is an associated one.

Dr Kahryn Hughes:

And then finally, there’s something which is that however rich the data are, they may not be able to inform the question or they may have limitations. Anna and I looked to see if we could build intergenerational samples by bringing two data sets together that had been conducted with people in similar socioeconomic circumstances, in the same geographical locality. Could we build a longitudinal sample of people that some of these people might be a sort of parent. Data proxy, if you like, a qualitative data proxy parent for the younger people in the study, and we couldn’t. We couldn’t. We needed more data in order to build those samples.

Dr Kahryn Hughes:

Qualitative secondary… we could build intergenerational samples, that’s absolutely perfectly feasible, but not intergenerational ones. The data themselves may have limitations in terms of how they can inform the questions that you have. You may have to do more research.

Leah Fullegar:

Does everyone feel prepared? Because I can guide you. I can just hear the hundreds of PhD students out there imagining how much more they can make from their PhD data, myself included, if I knew they had the funding and the time. Actually, that’s a good time to remind everyone that Dementia Researcher publishes all the dementia research funding calls. So it’s a good idea to head over to their website and take a quick look to see if there’s anything that fits your work.

Leah Fullegar:

Lastly, we come to the part of the show where we invite our eminent expert to have one minute to tell our listeners what they should go away and read to further their knowledge on this method. Kahryn, over to you. I’ll start the timer.

Dr Kahryn Hughes:

Well, Jennifer Mason’s special issue 2007 in sociological research online. I think that that’s a really great place to start for anyone. Go and have a look. It was a real field changer. On the Timescapes Archive, there are a number of papers and working papers and guides and so and so forth that were produced by Sarah Irwin and Mandy Winterton on QSA based on the work that they did during the Timescapes program. Absolutely fantastic, really accessible. And also, with some references to other papers, general papers that they also produced.

Dr Kahryn Hughes:

And slightly embarrassing, a quick plug for my own work with Dr. Anna Tarrant. We’ve written the first textbook on… well, not textbook. It is a handbook on qualitative secondary analysis. We have also written a large number of papers, blogs, and various other things on QSA. I’m a little bit squirmy saying that. I don’t know why, but I am. Yeah, just go and have a read.

Leah Fullegar:

You looked incredibly uncomfortable for the last bit.

Dr Kahryn Hughes:

I know. I don’t know what it is, but it’s like ooh.

Leah Fullegar:

Thank you, Kahryn. That’s fantastic, and a lot more to add to my reading list. And thank you very so much for finding the time today to join us. I’m afraid that brings us to the end of the series. I’m going to go cry now. These shows have been released each day, but they were actually recorded over several weeks and it’s been a fascinating journey that I really wish I had done at the beginning of my PhD rather than at the end.

Leah Fullegar:

Alongside the podcast, we’re delighted to be sharing some awesome posters and visual guides produced by the brilliant artist, Jack Broadbent. If you didn’t have time to take notes today, don’t worry, we have a great poster that shares all the highlights and everything you need to know. Take a look in the text below to download a copy or the whole series from the week. Print them and pop them on your wall. Thank you all so much for listening to this week, and thank you to the National Centre for Research Methods for all their support.

Leah Fullegar:

The NCRM delivers fantastic methodological training and resources on Qual and advanced qualitative quantitative digital creative visual mixed and multi model methods. That was a hell of a thing to say. Take a look at their website at ncrm.ac.uk for more information. If you’d like to more about qualitative secondary analysis, today’s guests, or any of the methods discussed this week, be sure to visit the Dementia Researcher website at dementiaresearcher.nihr.ac.uk. And remember, if you found this useful and learned some stuff, then please share this podcast with your friends or leave a review online. Make sure to subscribe to Dementia Research podcast in your favorite podcast app so you never miss an episode.

Leah Fullegar:

All that’s left for me is to say a huge thank you to all of our guests, particularly today’s. We’ve had the wonderful talented Dr. Anna Volkmer sharing her experiences. And in the expert corner, the incredible Dr. Kahryn Hughes. Thank you both, it’s been a pleasure.

Dr Kahryn Hughes:

It really has, thank you.

END