Researchers today have confidence in peer review, but strains on the system—including the increasing volume of research outputs and platforms to access research—are causing them to worry about quality and reliability, new research from Sense about Science and Elsevier finds.
Ten years on from the two orgs’ influential peer review survey of 2009, the new survey found that researchers don’t want to replace the peer review process, as 75 percent are satisfied with it (compared to 69 percent in 2009). But researchers do want to improve and extend the system. It’s a question of whenpeer review should be used and knowing when it has beenused. Over a third (37 percent)admitted they had doubts over the quality of at least some research outputs they had encountered in the last week.
“What this survey shows is that our inroads into getting wider understanding of peer review are now not enough for research publishing in the 2020s, or for people to understand what quality checks have been done,” said Tracey Brown, Sense about Science director, in a news release.
“We urgently need to see a common language and greater transparency about what has been reviewed. Does it include data? What criteria were used? Is it even a published study? Remember many research users are arriving at information via search engines, not academic portals,” Brown added.
Other results include:
- Respondents want more information available alongside research papers, including an indicator to show whether someone else had tried to reproduce the research (82 percent).
- Most researchers (76 percent) believe that data and supporting material that accompany research articles should be reviewed.
- Two–thirds of participants feel that reviewers would benefit from clearer guidance on reviewing criteria, while many felt that recognition of reviewing work was important for maintenance of a healthy peer review system, most commonly from employers.
- Few researchers (38 percent) agree that the public understand the concept of peer review. There is strong support (77 percent) for a clear indication that material has been reviewed; and most (70 percent) feel explaining research context, and the implications of research in lay terms would be most helpful.
- Despite the trend towards using technology to evaluate evidence, few respondents (8 percent) felt the evaluation that relied on artificial intelligence would qualify as peer review.
There are four areas for urgent action:
- Amid rising volume of research outputs and information platforms, researchers want improved signals, be able to cross-check work easily.
- Better training, information, and more career-based recognition is raised by researchers in this and other studies. Finding a way to remove the current inconsistencies in reviewer instructions is paramount.
- Agreement about the role of technology in peer review, e.g., using it to manage the rising volume of submissions and alleviate the burden on reviewers, without losing the benefits of human judgement? Without discussion, the use of AI may further disrupt people’s ability to trust content.
- Ensure that research is put in context: articles include easy-to understand explanations of research; and a common language is used to communicate to both researchers and public alike what has been reviewed and the type of review done.
“This study highlights the growing pressure being placed on research communication and importantly the value of peer review,” said Adrian Mulligan, research director for customer insights at Elsevier, in the release. “Maintaining the integrity of the peer review system is paramount so it’s important various stakeholders work together to ensure reviewers receive recognition, have clear guidance and quality tools to support them in their roles.”
In 2009, Elsevier and Sense About Science joined forces for one of the largest ever international researcher surveys on the topic of peer review. It asked researchers for their thoughts on aspects of the process; for example, what it should do, how it was performing, and how it could be improved. The insights became an influential reference document for publishers and policy makers alike. The new survey from Sense about Science and Elsevier included over 3,000 respondents, randomly selected from the Scopus database of over 3 million published researchers.
