Print This PostThere are many good reasons for open science. Noble reasons. Selfless reasons. It is to academia as the code of chivalry was to the knights of old (or at least the ones in movies). It is designed to make the research process more transparent using preregistration and open resources to aid the battle in making our research more honest and reproducible. But it also takes a lot of time, energy, and resources.
When I was first introduced to open science, I was a wide-eyed first year PhD student with a lot of enthusiasm for doing science and doing science well. But the more I engage in these practises, the more I realise that pursuing open science is also about self-interest. Thusly I’d like to dedicate this blog not to the noble but instead to all the selfish reasons you should implement open science so that you (like me) can be a rebel.
Number 1 – Preregistration makes me feel faster than Sonic the Hedgehog on a speed boost. Preregistration is the practise of stating your rationale, methods and hypotheses before you begin data collection or analysis. You can do this on free servers like the Open Science Framework which give you free reign to upload either incredibly structured outlines or more freeform narrative explanations of your intentions. A common pushback I see to this process is that it takes a lot of time and energy to do. As scientists we want to get to the actual science as quickly as possible and that can often manifest as analysing first and asking questions later. I would argue that preregistration rarely creates more work, it simply pushes work to the beginning of a project allowing you to clearly think through your methods and analysis ahead of time and without any bias from already knowing your data. Preregistration may have pushed my project back by a month or so at the beginning, but when I came to analysing my data, I did it in a day. I had been working on the code ahead of time to fit with the preregistration so by the time my data collection was complete, all I had to do was press ‘run’. This was, as you can imagine, hugely satisfying.
Next came writing, and most of my paper was ready to go. My preregistration already contained an introduction, methods, and hypotheses section which provided the skeleton for my results. Even now that I’m working on larger projects, I see comparable benefits. I may not be able to analyse all of my imaging data in a day, but what I do have is a framework laid out which gives me a step-by-step guide of how to run my analyses. All the mental capacity I spent planning at the beginning of the project, now pays dividends in the form of clear set of instructions that I simply have to follow.
Number 2 – Preregistration stops me from being sneaky by mistake (or on purpose). Another common pushback I see to preregistration is the classic: “But what if I have to change something?”. We live in fear as scientists that we will lay out an analysis strategy, find out that strategy doesn’t work, and then be stuck. To this I would argue that the goal of preregistration is not to tie your hands. If you need to change something in your methods or analysis because you realised something you didn’t know beforehand, then that’s fine. The difference is that now you have to explain why something has changed. If you have a valid scientific reason for shifting your strategy then that isn’t a problem, but if you find yourself without a good reason then maybe its best that you didn’t shift at all. It’s so tempting to try different analyses when your initial hypothesis doesn’t get that all important significant result, but if you’re trying new parameters just to hit that magic p-value, then you’re better off not changing things at all! I have been ‘caught out’ by preregistration in that I preregistered a primary analysis strategy that didn’t work. Functionally what this meant is that my paper featured a null result, followed by the working strategy and an explanation of why one worked, and the other didn’t. All the preregistration did was ensure that my null result couldn’t be swept aside without explanation, and I didn’t have to fight with co-authors to include it because there simply wasn’t a choice.
And on the topic of null results…
Number 3 – Open science makes publishing null results easier. Going a step beyond preregistrations to registered reports, you can submit your introduction, methods and hypotheses to a paper who will choose to accept or reject your work regardless of your results. No more publication bias based on significance, no more bottom drawer effect. And even if you don’t want to go that far, with the dawn of preprint servers like bioRXiv and medRXiv there just isn’t a reason for any results based on good science to be stuck unpublished. You can bypass the journals still get your work out there. You can even do it for free!
But that’s not all that comes as a free perk with open science, because getting your work out there can also mean getting your data and code out there. So we reach, number 4 – Free sanity checks. A lot of people (and by that, I mean me) worry that we’ve made mistakes in our analyses. And I think a lot of the fear of making our materials openly available is that someone will find one of these mistakes. This may seem terrifying, but in actual fact is exactly what we want. We all make mistakes, even us amazing scientists, and by making our resources publicly available we get free code checkers who might spot them.
There are countless noble reasons to engage with open science. Scientists who preregister, make their resources openly available and talk transparently about their research to other academics and the public are the shiny knights of academia.
But what no-one except us reading this blog need to know is that beneath said shiny armour is just a scientist who recognises that open science also has a lot of selfish benefits built right in.
So, if you’re on board for making your analyses quicker and writing less effortful, for no more fear of null results and free sanity checks on your code and data… then fly the open science banner high!
I also talked about this topic recently in the Dementia Researcher Salon – check out the recording.
Rebecca Williams
Author
Rebecca Williams is a PhD student at the University of Cambridge. Though originally from ‘up North’ in a small town called Leigh, she did her undergraduate and masters at the University of Oxford before defecting to Cambridge for her doctorate researching Frontotemporal dementia and Apathy. She now spends her days collecting data from wonderful volunteers, and coding. Outside work, she plays board games, and is very crafty.
Dr Becky Carlyle
Dementia Researcher
Dr Sam Moxon
Rebecca Williams
Emily Spencer