Blog – A few thoughts on peer review

Reading Time: 6 minutes

24/11/2020

From the moment you decide to follow a career in academia, or start your PhD, the pressure to publish your research findings starts – this is a blog about the peer review process.

The focus on the publication is often perceived as more valuable than the journey that got you there. If you are really lucky, that publication will result in something, or at least add to the body of research.

Bringing this back to basics, what are we talking about? Being funded, to discover the answer to a question, and then telling everyone what the answer was – even if the answer wasn’t the one you were hoping for!

As a PhD student, you are expected to independently (or semi-independently) conduct original and significant research, and then produce a publication-worthy thesis (or a thesis that leads to one or more publications). While some Doctorates include taught components, PhD students are almost always assessed on the quality and originality of the argument presented in their independent research project.

As a research funder, you fund people. People to answer questions you think are worth answering. People with clever ideas, who have interventions to test and experiments to undertake. To make discoveries. Funders want to be able to assess if their investment is paying off! There are many ways this is done (rightly so) but publication usually tops the list.

Okay, I can hear you saying “Get to the point already… we know this, and you’re depressing me”

So, there you are! You work for years, to uncover a small new truth. You write the data into a story, include evidence, citations, and conclusions. You get your colleagues or supervisor to look, and it makes sense. You carefully select the right journal, and you submit the paper for publication. The moment you do, you probably get another set of feedback (sorry too late), you may discover a typo (on the 150^th re-read through – because you always read it one last time after submitting) and you fully expect a rejection, but hope to see your name in print.

Then you wait… and wait… anything from two weeks, to two months or even longer! Then finally, you receive a reply. That reply, includes feedback from the peer reviewers.

I promise, I am getting to the point now

The reason for my rambling start to this blog, was really to make the point – that at this stage you have worked for years to perfect that paper – thousands of pounds, thousands of hours, stress, elation, pain and sacrifice. You only hope that on that basis, the three reviewers who were given the honour of being the first people to review your research, gave the manuscript the focus it deserved. That the journal selected people who had time, and expertise, and a passion for the process.

You nervously login to the online system (that you have been checking for months), to download the feedback.

What do you find?

“Could do with more experiments” – but you ran out of funding, and have moved on
“Add more information / explain” – which you already did, and what about the word count
“Reviewer 1 & Reviewer 3 contradict each other”– so what now?
“I disagree with these findings, or think you’re wrong” – based on what?

Or… even worse, they simply say “this needs work”, and at that point, you look for the wine, and turn to twitter.

The standard of peer reviewed is a common complaint among scientists. There are a number of twitter accounts dedicated to sharing those comments, see @YourPaperSucks, @thirdreviewer and @AcerbicAcademic or #Reviewer2.

Peer review is important. Even people who hate it, agree that is has merit. But I think we have a problem. The increasing number of publications, and digital journals have driven a culture of “publish or perish”, this means that not every paper really gets the review it deserves.

Everyone I know that has become a reviewer (including myself) says that being on the other side has given them a new appreciation of the challenge. I am not making excuses for them, but when you’re given unrealistic deadlines, doing the work for free, and asked to review things that don’t quite turn out to be what you were expecting, it is hard (although there is never any excuse for any off the cuff comment such as “The authors use an unnecessarily complex method where a simpler one would have sufficed” or “The arguments in the paper are compelling but not convincing”).

So, what is my point?

I think that there are lessons for everyone here:

Researchers, make sure your manuscript is awesome! As good as it can possibly be and get some input from people who are not the authors.
Journals, get better! Put more focus on finding appropriate reviewers, give them proper support and guidance, and recognition for their work, and the time to do a good job.
Reviewers, learn to say no. If you really don’t have time or the knowledge, just pass! If you do agree to review, do the best job you can. Remember that the manuscript you’re looking at involved a lot of work, so at least be constructive and respectful, and follow the guidance.

For those of you reading this, do not be deterred from becoming a reviewer yourself. This guide from Taylor & Francis does a good job of setting out the reasons why you should, in a nutshell:

It benefits your own publications and writing
You get to know the latest developments and ongoing research in your field.
It enhances your scholarly, research and teaching skills.
Your connection with your field gets highlighted.

But before you do, consider taking a course, and become as much of an expert at this process, as you are at your own writing, and the people whose research you are reviewing.

To make it easier, I found you a few:

And ignore the academic that, when questioned about the value of peer review training, said “Reviewers are sensible, intelligent people and they can interpret that piece of paper without further training. It’s just a waste of time”.

Big thank you to Beth Eyre for taking the time to review my blog, and provide constructive and helpful feedback in a timely manner 🙂

Author

Adam Smith was born in the north, a long time ago. He wanted to write books, but ended up working in the NHS, and at the Department of Health. He is now a Programme Director in the Office of the NIHR National Director for Dementia Research (which probably sounds more important than it is) at University College London. He has led a number of initiatives to improve dementia research (which happens to include creating this website, Join Dementia Research and ENRICH), as well as pursuing his own research interests. In his spare time, he grows vegetables, builds Lego and spends most of his time drinking too much coffee and squeezing technology into his house.

Follow @betterresearch