Detecting and Preventing Paper Mill Publications at the Royal Society of Chemistry

What Is a Paper Mill?

In January 2021, the Royal Society of Chemistry (RSC) retracted 70 articles believed to have originated from a “paper mill”—an organisation that systematically produces fake research for sale.[1] This was the result of a year-long investigation into a number of papers with very similar structures and content despite having no common authors. These articles often appear legitimate when viewed on their own, and concerns are only obvious when many similar articles are compared.

Paper mills prey on the “publish or perish” culture prevalent in some parts of the world.[2] In China, medical doctors are under immense pressure to publish in order to secure funding or promotion, despite having little time to actually carry out research themselves.[3] Researchers in many regions are incentivised to publish as many articles as possible, with numbers of publications being used in funding and promotion decisions. Paper mill companies produce fake data and images, and compile these into manuscripts that researchers can buy authorship on, often paying a higher rate for first or corresponding author positions.[4] For those desperate enough it is a quick and easy way to increase published output.

The RSC’s Investigation

At the RSC we first became aware of suspected paper mill manuscripts thanks to ScholarOne’s Unusual Activity Detector, which detected multiple manuscripts with different authors that had been submitted from the same computer. Working with some of our Editors on RSC Advances, our Publishing Ethics team carried out an extensive investigation, consulting with experts in image and data manipulation, and informed heavily by the work done by research integrity consultants such as Elisabeth Bik.[5]

In late 2020 COPE (Committee on Publication Ethics) held a forum discussion on paper mills’ systematic manipulation of the publishing process.[6] This was very timely for our investigation, and helped us ensure we were taking all the right steps.

After receiving a list of articles that had triggered the Unusual Activity Detector, our first steps were to triage all the articles on this list against a checklist of common paper mill characteristics. We also searched our archive for related articles. We ruled out any articles that didn’t meet our criteria, and put together a shortlist to investigate further. Following COPE’s guidelines we requested raw data from authors, alongside explanations for any instances of image duplication or manipulation we’d spotted during the triage. We then checked for text overlap using iThenticate, checked for errors in nucleotide sequencing using the SeeknBlastn tool,[7][8] and consulted an image integrity specialist, who analysed all the raw data and images. The whole investigation took about a year to complete.

The result was the retraction of 70 articles, and an ongoing commitment to detect and retract any more we may have missed. Each retraction notice was individual, clear, and detailed, and all authors were contacted in advance and given an opportunity to provide raw data and prove the validity of their results. In the majority of cases, we received no response at all from the authors, and when we did there was a surprising—or perhaps unsurprising—amount of similarity in the appearance and wording of these replies, often using identical phrasing, or with data being formatted and labelled in the same pattern. This only added to the suspicion, and similar things were experienced by other publishers carrying out similar investigations.[9][10]

Moving Forwards

The ultimate goal for publishers and researchers alike is preventing the existence of paper mills in the first place, reducing the “publish or perish” culture, and ensuring the emphasis is on quality and thoroughness of research rather than quantity or perceived impact. New Chinese research assessment guidelines announced in early 2020 go some way towards tackling this culture, encouraging authors to focus on quality rather than quantity, and only assessing a smaller number of “representative works” when it comes to funding and promotions.[11] These new guidelines align with initiatives such as the Declaration on Research Assessment (DORA)[12] which argues for fairer methods to assess research outputs, with less reliance on metrics such as the Impact Factor. In China, cash rewards for publishing articles were banned in 2020,[13] and in late 2021 China’s ministry of science and technology announced sanctions against hundreds of individuals thought to have published fraudulent articles,[14] which may act as a further deterrent.

In the meantime, publishers like the RSC continue to work on improving detection and preventing publication, and we’ve taken a number of steps to ensure this:

• Introduced additional screening checks at submission—checking author institutions and email addresses; ensuring articles contain raw data; looking out for formulaic titles and common paper mill features in the introduction, discussion, and figures; checking manuscripts without funding information.

• Strengthened data requirements—updating our guidelines for Western blots initially, and subsequently for other data types; empowering our Associate Editors to reject manuscripts that don’t provide sufficient data; updating our data reporting and sharing guidelines.

• Increased awareness among staff—raising awareness of the existence of paper mills; sharing submission checking procedures with other journal teams; increasing awareness of the indicators of paper mills.

• Conducted training and awareness raising for Associate Editors—holding a training session for our academic editors; sharing examples of manipulated figures, formulaic titles, and other key characteristics; encouraging Associate Editors to request raw data, and giving guidance on actions to take if they find suspicious manuscripts; appointing reviewers with experience in detecting image and data manipulation.

• Trained staff in detection of image manipulation—providing training for members of our ethics and journals teams; empowering our staff to run their own checks and to assist our Associate Editors in spotting image manipulation; sharing knowledge.

• Piloted Proofig image manipulation software—after trialling this image-screening tool, rolled it out across our journals; working with Proofig to provide feedback and improve tool.

The final piece of the puzzle is collaboration. We wouldn’t have been able to identify and retract these articles without tools, advice, and information from a variety of sources and people, and now that we have gathered all this knowledge, we want to share it as widely as possible. We’ve had discussions with various publishers looking to repeat the exercise on their own journals, and have always been happy to share any information we can. Many other publishers have subsequently announced similar retractions, and we expect there are yet more to follow.[15][16][17][18] Meanwhile, image manipulation experts and research integrity sleuths continue to do great work in identifying new patterns and ways of detecting research fraud.[5][19][20] The publishing industry is working together to tackle the issue of paper mills. In December 2021, COPE revised their ‘Systematic manipulation of the publication process’ flowchart.[6] STM are working with a number of publishers, including RSC, to develop the STM Integrity Hub, which will “provide a cloud-based environment for publishers to check submitted articles for research integrity issues” and will give publishers an additional tool to help detect potentially fraudulent submissions.[21]

While the issue of paper mills comes from a fundamental problem with pressure to publish, the solution lies in all of us as publishers and researchers sharing our learnings and tackling this problem together.

Disclosure Statement

The authors declare that they have no conflicts of interest.