At the 2023 ISMTE Global Event, I spoke about the Standard Terminology for Peer Review in a session entitled ‘Creating agreed standards in journal publishing.’ This article builds on that presentation and provides an update on progress since then.
By way of historical context, although the term ‘peer review’ dates from the 1970s1, the concept of consulting learned experts for an opinion on someone’s research has existed for far longer. Readers will doubtless be familiar with the oft-cited criticisms of peer review in the journal context: that it’s outmoded, ineffective, expensive, time-consuming, or any combination of all of these. Criticisms of this nature are hardly new, however, and one of my favorite quotations on the topic comes from the biologist Thomas Jukes (1906–1999):
It has been said that the warp that holds the complex fabric of science together is peer review, and the woof is the noise made by scientists who complain about it.2
In the early 19th century, Augustus Bozzi Granville, a fellow of the Royal Society with the curious distinction of having undertaken the first-ever autopsy of an Egyptian mummy, critiqued how papers presented to the Society were evaluated by the fellows:
At many of these meetings, members of the committee of papers have been present who have not the smallest pretension to any knowledge whatever of the subject under consideration.3
There’s nothing new under the sun, as a sage wrote centuries ago (Ecclesiastes 1:9).
One could argue that most such criticisms are levelled at a single, narrow definition of ‘peer review’—typically, single-anonymous and pre-publication—whereas in practice it is far from homogeneous. Double-anonymous review is the most obvious and well-established variation, and in recent years we have witnessed the emergence of new experimental models, facilitated through new technology and largely driven by the push for greater openness, in parallel with the push for open science, open data, and open access. The recent publication by the Royal Society of 80 volumes of referee reports dating 1832–1954 is itself a vigorous nod towards greater transparency, as well as providing a fascinating historical resource for anyone with more than a passing interest in the development of scientific knowledge.4
Different types of peer review seek to mitigate, if not eliminate, the weaknesses of the single-anonymized, pre-publication model. For example, a double-anonymous process helps to reduce the risk of bias, and an open process whereby reviewers’ names are visible to authors aims to increase accountability. Technology supports the integrity and speed of the process through features such as automated screening checks and reviewer search tools, as well as providing opportunities to experiment with improving efficiency through workflows such as community or ‘crowd’ peer review5, post-publication commenting6, pre-publication interaction between author and reviewer7, and journal-to-journal peer review transfer.8
This landscape of multiple models of peer review can be overwhelming, however, and until recently there has been no consistent way of defining and describing all the different dimensions. To take one example, a systematic review in 2017, for example, suggested there are at least 22 definitions.9 This smorgasbord of features creates confusion for researchers: as reviewers, they are expected to know the model for any journal for which they review; as authors, they want to understand what they can expect from the review process once they’ve submitted their manuscript; and as readers, they need to be able to trust the process of scholarly scrutiny to which an article has been subjected. One of the hallmarks of a credible journal is that it is transparent about its review process.10
It’s therefore vital that journals publicly describe their peer-review process and do so in a way that is consistent within the scholarly domain: and this is the raison d’être for the Standard Terminology for Peer Review. Originally named a ‘Standard Taxonomy for Peer Review’, it derived from a Working Group of the STM Association established in June 2019 under the leadership of Joris van Rossum. A small number of publishers piloted the taxonomy in 2021, and, after feedback, ‘Taxonomy’ was altered to ‘Terminology’, which is strictly more accurate. The Working Group approved the final version in early 2023, and in July 2023 it was formally approved and adopted as standard Z39.106-2023 by the American National Standards Institute (ANSI) and National Information Standards Organization (NISO).11 Since September 2024, the Standard has had a dedicated website.12 As can be seen from the infographic illustrating the Terminology (Figure 1), it has four dimensions:
1. Identity transparency: what level of visibility or anonymity is there in the process?
2. Reviewer interactions: are these solely with the editor, the most-common model, or do they interact with other reviewers and/or with authors?
3. Published review process information: this might include publishing the actual review reports, the decision and response letters, names of reviewers and editors, or any combination of these.
Implementing the Terminology is as simple as displaying the elements of the peer-review process on a journal website using the nomenclature provided (Figure 2).
Figure 2: This illustrates how one journal, Physiological Entomology, includes the Terminology within its author guidelines.
It’s important to note that the Terminology is purely descriptive: it does not make any value judgments about any particular model or process. In describing different elements of the peer-review process, it implicitly recognises that the process can in practice be complex. For example, a journal might operate both a double-anonymous pre-publication process and publish review reports post-publication; or a journal might publish review reports with or without reviewer identities. Although it is initially intended to be descriptive at a journal level, the intention is that it will be developed for implementation also at the article level.
In addition, the Terminology in its current form does not tell the full story about the processes that an article goes through to reach publication. It does, however, encourage publishers to include other peer-review information, such as:
Journal-to-journal transfer processes
Submission, acceptance, and publication dates
Whether the manuscript was fast-tracked
Number of reviewer reports
Number of rounds of revision
Technical tools used in the editorial process such as plagiarism checks or tools to detect data manipulation
Additional information to explain parts of the process; for example, it is helpful to state if reviewers need to register in order to be able to comment post-publication
While at present the Standard is intended for journals, it may in the future be expanded to include other publication outputs such as books or data, and it may also include guidance on machine readability which could be valuable for people doing research into peer review and research evaluation.
Since the publication of the Standard in 2023, a Standing Committee has been formed to continue supporting the work, providing publicity and further information, answering any questions, and serving as a general community resource for adoption. A list of members of the original Working Group and the present Standing Committee is available at https://peerreviewterminology.niso.org/about/.
Our ambition is for as many publishers and journals as possible to recognize the value and importance of being transparent about peer-review processes, for the benefit of researchers as authors, reviewers, and readers, and to adopt and display the Terminology on their websites. The website provides guidance (https://peerreviewterminology.niso.org/getstarted/), including a one-page downloadable file summarizing the key points that can be shared with editors and other relevant stakeholders. If you have implemented the Terminology or are in the process of doing so, please contact [email protected] so this can be added to the dashboard (https://peerreviewterminology.niso.org/dashboard), and also consider providing a testimonial to encourage others to adopt the Terminology (https://peerreviewterminology.niso.org/testimonials).
References
1. Moxham N, Fyfe A. The Royal Society and the Prehistory of Peer Review, 1665–1965. Hist J [Internet]. 2018 [cited 2024 Oct 25];61(4):863–89. Available from: https://www.jstor.org/stable/26799973
2. Jukes TH. Peer review. Nature [Internet]. 1977 Jan [cited 2024 Oct 25];265(5591):203–203. Available from: https://www.nature.com/articles/265203b0
3. Granville AB. Science without a Head; or, The Royal Society Dissected [Internet]. London: T Ridgway; 1830. Available from: https://archive.org/details/b3187180x
4. Royal Society Archives: Referee Reports [Internet]. Royal Society; [cited 2024 Oct 31]. Available from: https://catalogues.royalsociety.org/CalmView/TreeBrowse.aspx?src=CalmView.Catalog&field=RefNo&key=RR
5. List B. Crowd-based peer review can be good and fast. Nature [Internet]. 2017 Jun [cited 2024 Oct 31];546(7656):9–9. Available from: https://www.nature.com/articles/546009a
6. Lechner I, Hosseini M, Mezinska S, Perković Paloš A, Amin E. Post-publication peer review. In: Embassy of Good Science [Internet]. 2022 [cited 2024 Oct 31]. Available from: https://embassy.science/wiki/Theme:9025f215-cc6a-4b00-894b-68b9a089f173
7. Ross-Hellauer T. Defining Open Peer Review: Part Two - Seven Traits of OPR [Internet]. 2016. Available from: https://www.openaire.eu/blogs/defining-open-peer-review-part-two-seven-traits-of-opr-1
8. Match Your Research to the Right Journal with Refer and Transfers [Internet]. Wiley; Available from: https://authorservices.wiley.com/author-resources/Journal-Authors/submission-peer-review/manuscript-transfer.html
9. Ross-Hellauer T. What is open peer review? A systematic review [version 2; peer review: 4 approved]. F1000Research. 2017;6(588).
10. Think.Check.Submit [Internet]. [cited 2024 Oct 25]. Available from: https://thinkchecksubmit.org/resources/peer-review/
11. NISO MECA Working Group and Standing Committee. ANSI/NISO Z39.106-2023, Standard Terminology for Peer Review [Internet]. NISO; [cited 2024 Oct 25]. Available from: https://www.niso.org/publications/z39106-2023-peerreview
12. Peer Review Terminology [Internet]. Available from: https://peerreviewterminology.niso.org/
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
