Open science and pre-registration, why am I writing a blog about this? I often hear that PhD’s perceive their work as stressful, competitive, solitary and disconnected with the rest of the world. A seemingly unrelated problem is the private profit maximization and the collective inefficiency in information asymmetries that are inherent in competitive markets. What do these two problems have in common. Well maybe the solution: open science. The scientific world (or market?) is competitive, researchers work too much in isolation and care too much about their own reputation instead of developing knowledge for the public. This creates an environment where you earn credits for developing (or should I say creating?) ‘new’ and innovative findings. In order to develop these new findings, researchers are stimulated to keep the research design to themselves, so others cannot ‘steal’ your ideas. This results in a stressful context where colleagues compete instead of work together. What can open science do for us?
What is open science?
Open science is often described as a movement to make scientific research, data and dissemination accessible at each stage, thus not just through publishing in a journal. I prefer it as a series of steps you can take to increase the transparency of science. Open science consists out of several aspects:
- open access: distributing research outputs online and free of costs or other barriers (for instance, https://osf.io/).
- open educational resources (OER): sharing lessons plans and courses online (for instance, https://www.oercommons.org/ ).
- open source: publishing your syntax in an understandable manner.
- open data: sharing your data alongside your findings and allowing others to use your data.
- citizen science: collaborating with non-scientists).
Another important aspect of open science is that scientist are allowed to retry studies and not just to try new things. This is an important aspect to develop science build on robust findings: we should focus on reproducing findings.
Why is open science important?
The VU University is aware of the need for a more open research approach. In November 2018 the University Library of the VU University organized a seminar focused on open science. The next meeting will be January 24 (15:00 in Aurora, head building: https://ub.vu.nl/en/education-research/library-live ).
During this seminar Rene Bekkers (2018) argued that the transparent nature of open science can reduce the temptation of private profit maximization and increase the efficiency by working together instead of against each other (that is competitive science). Another reason why open science is important is to support your fellow researchers. How you ask? It is well known that non-significant results are harder to publish (that is the publication bias, (i.e. Scherer, Langenberg, & von Elm, 2007), and as a result harder to find. If we pre-register and publish non-significant findings we can ensure that other researchers are aware that a certain relationship doesn’t exist. This way we can decrease the file drawer effect: not publish negative data and filing it away without telling anyone.
A focus on open science is also expected to decrease p-hacking. In the case of p-hacking a researcher misuses data-analyses to find certain (desired) patterns in data that can be presented as statically significant, while actually there is no real underlying effect.
In short, we can summarize the advantages of open science in four keywords:
- Increased access: by sharing the data and codes (i.e. syntax) online, researchers make the information more accessible for everyone. This way the data can develop knowledge an serve public goods.
- Higher efficiency: collecting your data takes a lot of time. If we share our data more regularly it becomes easier to conduct research: we can build on the work of others.
- Increased quality: by sharing the steps you took in order to find the results, you allow others to comment on the whole process. This way you can receive more in dept feedback. In other words, we enable other to check all of our work and not just publishable results. This way we can improve the quality: others can help you develop the best way to analyze the data.
- Knowledge: the ultimate goal of science should be developing knowledge. The increased access, higher efficiency and increased quality are likely to result in more knowledge.
Some friendly advice: DOI your data when you share it. DOI means that you give the data a Digital Object identifier. This way others can understand where the data comes from and where to find it themselves. For more information you can check: http://help.osf.io/m/sharing/l/524208-create-dois
What is a pre-registration and why is it important?
As a PhD you are often asked about the number of publications you have. In this blog I want to talk about a step before you publish your study, namely preregistering your design. As a researcher you have the option to share your design plan before you collect your data and publish your study. In this plan you specify how you intend to carry out your research, such as: the main research question asked and hypotheses, the key dependent variable(s), how many conditions your design will include, which analyses you will conduct, secondary analyses, the number of observations/sample size. You should write (and publish) your plan before you collect any data. A platform that can help you write a pre-registration is AsPredicted (https://aspredicted.org/ ) or OSF (https://osf.io/prereg/ ). After you have written the plan, you can upload your pre-registration to an archive like OSF (https://osf.io/ ). You can also include a link of your pre-registration in your paper. Another possibility is to distribute it through social media, like Twitter. I can definitely advice this: it provides you with the opportunity to receive feedback before you collect your data.
Some researchers argue that journals should accept papers solely based on pre-registrations, no matter the outcome (significant or not, positive effect or not). We should learn from failures to confirm a hypotheses and publish null findings.
How am I connecting with the world?
It takes a long time before you publish (if you publish) your first paper and that is ok. You should take your time, to develop a good design build on an elaborate theoretical framework with well defined hypotheses, analyze your data, present it at multiple conferences (preferably also ones outside of your field to develop a broader perspective). And indeed this can be lonely process, but it doesn’t have to be! In the meantime, you share your ideas and problems online in order to receive feedback from your peers.
I make the findings, study materials, analysis code, and data openly available
I work for the center of philanthropy at the VU University. At this center we house one of the biggest philanthropic studies: Geven in Nederland. This research is a survey based data file collected biannually among households in the Netherlands. The research is essential for the philanthropic sector and used around the world. Our center is an absolute for stander of open science and we promote our findings online before and after publishing. This means that we do not only share our findings, but also the data. Anyone can use our data and codes in order to conduct their own analyses.
I often share insights through Twitter (@PSCTeunenbroek), LinkedIn and my personal site: https://crowdfundingpscvt.wordpress.com/publications/ . I also share my research designs though a preregistrations (as done with my previous study http://aspredicted.org/u5w9u.pdf), and the data (for instance https://mfr.osf.io/render?url=https://osf.io/g8y2n/?action=download%26mode=render ) and syntax. In addition, I upload a draft/working paper version before sending it to a journal and after it has been reviewed (for instance https://mfr.osf.io/render?url=https://osf.io/emc35/?action=download%26mode=render ). This way individuals can keep track of the changes along the way. Also, I write practical blogs of my findings for a key online journal for philanthropic practitioners ( https://www.dedikkeblauwe.nl/news/claire-van-teunenbroek ), this way not only academics can use the findings but also practitioners.
Some useful tools to share your research at various stages of your study
There are several tools and platforms that can help you share your ideas, design, data, analyses, findings and conclusions. Below I have listed a couple of them.
- Communication and sharing: Twitter, LinkedIn and WordPress (personal site)
- Sharing posters and presentations: Figshare
- Sharing your drafts and preprint: OSF, arXiv or bioRxiv
- Sharing notebooks: OpenNotebookScience
- Sharing code/syntax: GitHub
- Sharing data: Dyrad, Zenodo or Dataverse
- Pre-registering your design: OSF or AsPredicted
- Sharing (grant) proposals: RIO
I hope I made you at least interested and curious about open science, it is an important topic. By embracing open science it is easier to connect with others. You can start by becoming (more) active on Twitter, this way you can share your ideas, questions and findings with your peers. Also, we as young researchers can make a difference by embracing open science and pushing the scientific world towards a less competitive world. So fellow PhD’s connect with the world and pre-register your designs, collaborate, share and publish in open journals.
Bekkers, R. (2018). Closing the age of competitive science. Presented at the University Library Vrije
Universiteit seminar, November 1, 2018, Amsterdam. Retrieved from: https://renebekkers.wordpress.com/2018/11/03/closing-the-age-of-competitive-science/
Scherer, R. W., Langenberg, P., & von Elm, E. (2007). Full publication of results initially
presented in abstracts. Cochrane Database of Systematic Reviews.
For more information about open science in Dutch: https://www.openscience.nl/