It has typically been a long and sometimes difficult process to notify researchersabout mistakes in published articles. This year’s institutional winner of the Einstein Foundation Award, the online platform PubPeer, has sought to make this process quicker and easier by placing discussions about article quality in a public forum, strengthening research integrity.
 

Once an article has been published in a highly regarded journal, it is considered indisputable for many researchers. This view of scientific knowledge production is widespread. But reality shows that it is worth taking a closer look. The online platform PubPeer has made it much easier to raise concerns about published academic research. Prior to its launch, the only way to do this was formally, via an official letter to the editor of the journal in question, or informally, through methods such as posting on social media or blogs. Often, this was a dead end.
 

PubPeer allows everyone to post anonymous or signed comments about research that has been published in the scientific literature, open a discussion on its content and integrity, and point out shortcomings or commend good methodology. Any article with a Digital Object Identifier (DOI) can be reviewed. The site has gone from strength to strength and has already logged 57,600 comments so far this year, compared to 2129 in its first full year after its launch, with over 300,000 comments recorded in total since 2012.
 

“It is crucial that the scientific community gets a grip on fraud, and without PubPeer that would be much harder.” (Dorothy Bishop)
 

It is estimated that since PubPeer’s inception, 19 percent of all retractions of papers worldwide in all academic domains had a prior discussion on the site. The site has also helped to raise concerns in some much-publicized fraud cases. Perhaps the first case that brought PubPeer to the attention of a wider audience was that of Olivier Voinnet, a highly-regarded researcher whose work, from his PhD onwards, was discovered to contain many manipulated images.
 

“Fraud used to be regarded as a very rare problem, but over the past 10 years or so, it has become clear that it is a very serious issue, not least because of the growing number of so-called paper mills, criminal companies that sell authorship and use illicit methods to get articles published,” commented Dorothy Bishop, Emeritus Professor of Developmental Neuropsychology at the University of Oxford, who nominated PubPeer for the 2024 Institutional Award. “It is crucial that the scientific community gets a grip on fraud, and without PubPeer that would be much harder. Having discussions out in the open makes a big difference to the quality of scientific debate.”
 

PubPeer was initially founded as an anonymous corporation by neuroscientist Brandon Stell, co-leader of the Cerebellar Neurophysiology team at the Centre National de la Recherche Scientifique (CNRS), based at Université Paris Cité. Stell set up the site with the help of developer George Smith and his brother Richard Smith.
 

“It was frustrating that evaluation of research scientists seemed to be based entirely on where they had published.” (Brandon Stell) 
 

“The idea for PubPeer came in grad school, where we had journal clubs to discuss newly published articles. It was clear that articles were getting a lot of respect based on the journal that they were published in. We would discuss papers published in highly ranked journals, and we would often find problems with the science or interpretation,” explained Stell.
 

“It was frustrating that evaluation of research scientists seemed to be based entirely on where they had published. I thought it would be great to capture some of these other conversations happening about articles and make them public. Scientists could then add a second layer of evaluation of the paper, regardless of where it was published.”
 

Although it was not the original aim of the site, it quickly became clear that PubPeer could be a useful tool for rooting out fraudulent research. A few years after the site was launched, a researcher identified on PubPeer as a perpetrator of alleged scientific fraud took PubPeer to court to try and identify an anonymous commenter on the site who had criticized their work.
 

The case was not successful, but it did prompt Stell and colleagues to consider the future of PubPeer. They decided to create a nonprofit called the PubPeer Foundation, with help from the Laura and John Arnold Foundation, to run the site. This was announced in 2015 and at the same time, Stell and the rest of the inaugural board came out from anonymity.
 

“The initial funding and formation of the Foundation allowed us to create a subscription service,” explained Stell. “Institutions, publishers, journals, or other interested parties can subscribe to receive alerts whenever one of their publications is commented on. That is what now funds the website.”
 

The PubPeer website has been criticized for its partly anonymous model, but Stell emphasizes that anonymity does not mean that comments are not rigorously moderated. “We have heavy moderation of the website. It’s a misconception that, because it's anonymous, people can leave whatever comment they want on the site, but actually we have a strict moderation policy,” he said.
 

“You can leave a comment in your name. If you do that, your first few comments are going to be moderated before they go live, but then you can be ‘whitelisted’ so that they are reviewed after they've gone live. It’s the same for an anonymous account.”

“It is important to underline that any utility or influence that PubPeer has earned is thanks to the community of experts that contribute to discussions on the platform.” (Brandon Stell)
 

Currently, there are at least 5500 anonymous and 1000 signed comments posted a month. Although the anonymous posters are still in the majority, Stell says they are starting to see a turn in the tide. “As the site has grown, the acceptance of commenting is gaining speed. A lot of people now want credit for their comments,” he explained.
 

“People that were really hesitant about the idea at the beginning, thinking that having anonymous comments on the site was going to be a disaster, now support us. I think that's a major accomplishment to show just how useful these comments are,” says Stell, “In general, it is important to underline that any utility or influence that PubPeer has earned is thanks to the community of experts that contribute to discussions on the platform.”
 

When asked about his goals for PubPeer for the future, Stell says he would like to see even more people in the academic community accepting the site and recognizing that it can be a very useful tool for maintaining research standards, promoting open discussion and building analytical skills. For example, he would like to see young scientists who regularly post helpful comments on the site being able to use this to their advantage when looking for jobs, in the same way that reviewing articles the more traditional way is regarded as beneficial for an academic career.
 

“This sort of award from the Einstein Foundation is just fantastic for PubPeer and its users, because it puts a little bit of a stamp on the site and gives a validation of its utility,” said Stell. “The prize money will enable us to accelerate the development and expansion of the site. For example, we would like to explore ways to support the creation of communities of site users.”
 

Stell added that he and the PubPeer team would also like to revisit the original goal of having more journal club like discussions on the site, less based on fraud, and more on scientific discussion and evaluation of newly published articles. “I think that body of evaluation could make a big contribution to science. It could help relieve the need to publish in non-open access journals and focus the evaluation on the content of the article instead of the journal title,” he explained.