It’s been a year since the Open Science Collaboration’s publication on “Estimating the Reproducibility of Psychological Science” was published in Science. It has been cited 515 times since publication, and has been met with much discussion on social networks.
I am interested in what changes your psychology department have made since. Are staff actively encouraged to pre-register all studies? Do you ask faculty members for open data? Are faculty members asked to provide open materials? Do ethics panels check the power of planned studies? Have you embedded Open Science practices into your research methods teaching?
I am preparing a report for my department on how we can address the issues surrounding the replication crisis, and I would be very interested to hear what other departments have done to address these important issues. Please comment on this post with what your department has done!
Jim Grange 
Hi Jim, good luck for your report and your initiative! It’s great that the issue gets attention in your department.
At LMU Munich, we made some noticeable changes; in short:
– foundation of an Open Science Committee (OSC) with all chairs represented. You can find some material of our OSC (e.g., the mission statement) here: https://osf.io/mgwk8/
– OSC meets twice each semester, offers workshops and makes suggestions to the departments steering committee. Sends regular newletters (see here, feel free to copy or resend it! https://osf.io/mgwk8/wiki/home/)
– OSC gives a quick report at every official meeting of the department’s steering committee
– We changed our formula for the internal distribution of the department’s money. Open data, open material and preregistrations now generate bonus points; i.e. reproducible researchers get more money (see http://www.nicebread.de/distribute-funding-based-on-research-transparency/)
– We added an “open science paragraph” to a professorship adverstisement (see http://www.nicebread.de/open-science-hiring-practices/). Soon we will advertise another professorship position with that paragraph.
– Ethics committee checks proposals for power (did it already before “the crisis”)
– All students learn about the replication crisis in at least one course/lecture
Furthermore, there are now 6 official open science initiatives at departments in Germany/Austria. We connect in the “network of open science initiatives” (NOSI), see https://osf.io/tbkzh/ (mostly German, sorry). In this network we share experiences (What did you already implement? What are the obstacles? Can i haz yur material?). This is very useful.
This is fantastic – thanks very much Felix!
Hi, Jim, I’d like to report the changes we have so far implemented at our department (Psychology, University of Koblenz-Landau, Germany). We are very proud to be one of the six official initiatives since we are a rather small department. Admittedly, we are not as advanced as LMU 🙂 – but we try very hard to keep up.
However, I think that the first and very important change was the foundation of an Open Science Committee itself that led to numerous enlightening discussions within the department. It was astonishing – and vexing – to realize that there are people out there, who do not even realize that something is screwed (just last week, one of our professors admitted at lunch that he never heard about p-hacking). Hence, I think that having a Committee emphasizes how important it is to have the time and an official space to talk about all that stuff.
So far, the Committee organized several invited talks (e.g., on power) and workshops (e.g., Felix was here). This month, we will have a workshop on preregistration, and we will organize our own preregistration challenge. You could call it a “curriculum on open science.”
Within the Open Science Committee, we are preparing suggestions for the department’s steering committee. However, I am rather pessimistic about our chances to change things like distribution of the department’s money or job advertisements. At least, in our group (teaching stats) we will call the attention of our students to the replication crisis. We made some small changes: for example since we do not teach SPSS anymore, we encourage our students to write reports in RMarkdown. In the courses on experimental design, the courses accompanying the bachelor theses, etc. we try to stress “good” science practices even more as before.
Overall, that is not much, but at least we are on track!
That all sounds fantastic! Thanks so much for sharing!! I will post an update on what we have done within our department once I have met with head of school to ask for changes to be made. I think it’s very important to get a committee within the department, and I think this seems to be a common theme.
Yes please, an update would be great!
My department has not officially done anything to address these issues. I know some colleagues are following this discussion, and some are not; at least one colleague that is following the discussion told me that s/he thinks all of this is just a “fad”. A few years ago I adopted open science practices in my own lab, and made changes to my graduate course on research methods in social psychology to focus more on open science and replicability (https://osf.io/nxytf/).
Thanks for commenting. I’ve realised that Twitter has lulled me into a false sense of security that everyone is aware of the replication issues and best practice to improve it, but the more I talk to non-Twitter colleagues I realise most are still unaware.