Maintaining Research and Publication Integrity

The reporting of scientific results has lately been under scrutiny. The number of retracted articles appears to be increasing, and discussions about compromised research integrity in scientific publishing have become a common occurrence in the lay press. These developments will undoubtedly have negative repercussions and undermine the credibility of the scientific community in the public eye. Historically, scientists have enjoyed the trust and respect of the public. The findings and conclusions of scientists' studies have resulted in beneficial changes in public policies and technical advances that have enhanced the well-being of society. The honest reporting of scientific results is the cornerstone of this trust and relies on the efforts and best intentions of publishers, editors, authors, and reviewers. These 4 entities must perform their duties honorably to preserve the integrity of the system. Unfortunately, financial incentives and greed, fierce competition among scientists, misguided criteria for academic promotion, and imperfect human nature have contributed to a questioning of the integrity of the system. The number of awarded PhD degrees has increased dramatically in the past 2 decades and so has the intellectual output. Accordingly, the number of scientific journals indexed by the Web of Science has increased by 66% since 2000, from 7383 to 12271. The increase in the number of published articles, per se, is not a problem. However, with the birth of predatory journals that have infiltrated the scientific publishing industry, the quality of the scientific output has been diminished. Although there is no definitive definition for a predatory journal, there are criteria, which when met, classify a journal as such, including the journal's editor and publisher being one and the same, having no editorial board or there is an editorial board but the members are not aware that they are listed as such, having no publications, and so on. These open access publishers are primarily based in South Asia, the Gulf region, Turkey, and Africa. According to an investigation by the German Public Broadcasters NDR and WDR, together with the Süddeutsche Zeitung Magazin, predatory journals have involved >400000 scientists globally. Furthermore, hundreds of thousands of articles are published yearly in these predatory journals: Indian authors are in first place as far as the number of published articles in these journals, and US authors are ranked second. Some of the American scientists who published in these journals were unaware that these were predatory journals; in fact, no comprehensive list of predatory journals exists. The US Federal Trade Commission recently initiated a lawsuit against Omics Group, the Indian publisher of 785 titles with over $50 million in revenue, for questionable practices. Combined, these issues have led to the proliferation of minimally important scientific publications, a cluttering of the literature with reports of little value, and a negative impact on research integrity. In this Q&A article, we discuss these important issues with an ethicist, an investigative journalist, a journal editor, and representatives from government and independent organizations that enforce and advise on issues related to research integrity.

What is the single major threat to research integrity?

Scott Moore: Research integrity is a broad area that encompasses many regulatory and ethics policies. It is difficult to identify a single threat to research integrity. For the Department of Health and Human Services (DHHS)⁸ Office of Research Integrity (ORI), our focus is a regulation (42 Code of Federal Regulations Part 93) that is the DHHS implementation of the federal research misconduct policy. Under this regulation our focus is research misconduct—fabrication, falsification, and plagiarism in the research record—with our Division of Investigative Oversight. In our Division of Education and Integrity, we develop materials and provide other support for the National Institutes of Health (NIH) requirement for Responsible Conduct of Research. There are 9 competencies expressed under NIH policy, including data sharing and management, conflict of interests, mentorship, authorship, and research misconduct. For a complete list, see https://grants.nih.gov/grants/guide/notice-files/not-od-10-019.html. From our perspective, we see complex relationships between the 9 competencies and the research misconduct allegations reported to us. Many real-life research misconduct cases have underlying themes involving lapses in several competencies, such as data management, mentorship, authorship, and conflicts of interests. There may be some value in looking at other misconduct models to describe the dynamics involved. One that we have been considering is the Cressey Fraud Triangle, borrowed from the world of white-collar crime. Fraud in that context has a different definition from research misconduct, but the sides of the triangle (opportunity to commit misconduct, an ability to rationalize doing so, and a perceived pressure) seem readily applicable to the research misconduct case. When all 3 are present, there may be an increased chance for misconduct to occur. Some of our education materials already are targeted at one or more of the sides of the triangle.

Arthur L. Caplan: The greatest threat to integrity in research is the failure to educate young investigators about their responsibilities. Laboratory directors, chairs, deans, and others must instill virtue in investigators if the integrity of research is to stand up to commercial pressures, outside influences, and the demands of tenure, promotion, and ambition.

Deborah J. Sweet: This is a multifaceted question, and the term research integrity means different things to different people. Several different areas were outlined in the introduction, and even that isn't a comprehensive list. For some people, the main issue is the conduct of research itself and data presentation; for others, reproducibility, and for others, transparency or conflicts. So, I cannot highlight 1 thing that is the single major threat. There are many potential challenges facing researchers and the research enterprise and many ways in which we can do better. However, I also would like to emphasize that I don't think the research system is inherently “broken” as some have argued. The vast majority of scientists pursue their research in an honest and straightforward way, with a high degree of integrity, and the scientific community continues to make exciting progress and push forward in many different areas. We don't have all the answers—and probably never will—but I think we do a disservice to science and its role in society if we give the impression that it can't be trusted.

Peter Hornung: Predatory publishers are certainly one of the major threats to research integrity. I can't say whether they are the biggest because we have limited our investigation to this phenomenon and haven't compared it with other phenomena that also harm science. Predatory publishers are certainly the most obvious threat, and they are probably the easiest to combat. They have been underestimated in their dangerousness because they have been seen as a problem of individual groups: young inexperienced scientists or scientists from the developing world or emerging economies. In fact, the latter are still the largest group of authors. Nevertheless, our findings showed the rapid growth in Western countries, where tens of thousands of authors populate this “parallel world” of predatory journals and conferences. The arrogance towards this problem, often combined with a patronizing undertone towards scientists of supposed or actually “lower rank,” may have made the problem of predatory publishers as big as it is today.

Frits R. Rosendaal: Research integrity, or a breach thereof, varies from sloppy science, to research waste, to outright fraud. While data fabrication or falsification receive the most attention, it is likely that they are rare. Sloppy science and research waste are more common. Sloppy science is research that is performed in a way that is far below the standard. The lack of adequate measures to minimize the risk of spurious results, e.g., by replication, can be seen as sloppy science. Research waste consists of studies that deal with irrelevant questions. Of course, this cannot be sharply defined, but articles that elicit the question “so, what?” from the reader fall under this category. These kinds of studies are at best a waste of research funds, and at worst may lead to spurious conclusions that can even do harm. There are all kinds of ideas about the causes of breaches of research integrity, such as publication pressure, closed environments, and intimidating superiors, but while these sound like common sense ideas, the truth is that we do not fully know at present.

What are the consequences of jeopardizing research integrity?

Scott Moore: Federal grants, including research grants, are awarded for public benefit. So, a lapse in research integrity is likely to jeopardize public (i.e., taxpayer) confidence in federally funded research. There is also the lost time and expense of following false leads and attempts to replicate work that was fabricated from the outset. There can be serious consequences for patients when medical treatments are based on falsified or fabricated data. For the individuals who engage in research misconduct, the consequences can be quite severe. A finding of research misconduct by ORI may result in a range of remedial actions tailored to the misconduct, including research supervision, grant termination, and exclusion from all federal programs (also known as debarment). At the institutional level, there can be a variety of employment actions taken up to dismissal or loss of tenure.

Arthur L. Caplan: Without integrity, subject welfare, both animal and human, is imperiled since researchers may cut corners with respect to protecting their welfare. Trust in findings will also erode if peers, funders, and the public come to believe that the values that undergird truth may have been corroded. Those who rely on data honestly reported to prescribe tests and treatments may find themselves at sea if integrity comes into question with respect to clinical research. And students will not find the requisite role models to ensure that the trustworthiness of the research process is incorporated into their behavior.

Deborah J. Sweet: This again is a multifaceted question, and therefore, it is not possible to come up with a single answer. The consequences depend on which aspect of integrity we are talking about. Inherently, though, if there is any issue that leads to a less than honest representation of research or its outcomes, that has the potential to be misleading. If that type of issue in turn leads to detrimental changes in real-world outcomes, then that is obviously problematic. Likewise, if questionable research is used to obtain funding or secure positions under misrepresented or even false pretenses, then that would be problematic as well. However, for the reasons I mentioned earlier, I think we need to take the approach that most scientists operate in an honest way and present their results in a straightforward manner. It's also important not to confuse integrity/reproducibility with the general progress of science. There are many examples of conclusions reported in a given paper that we later discover through additional study and analysis were incomplete or even incorrect. We expect this to happen, and it is just how science works, so while that type of situation can be frustrating, it doesn't mean there was anything wrong with the original article.

Peter Hornung: The credibility of science suffers damage. The reputation of scientists, universities, and research institutions also suffers. In the future, scientific results and findings could be under general suspicion if scientists fail to successfully combat the problem. Serious scientists make themselves common with researchers of dubious reputation, wannabe scientists, conspiracy theorists, or lobbyists when they publish in predatory journals and travel to the conferences of these predatory publishers.

Frits R. Rosendaal: There are 2 major consequences. The first is direct harm by incorrect results, a definite possibility in biomedical research. One example is the use of β-blockers in patients with cardiovascular disease who undergo major surgery. This was found to have a huge effect in reducing mortality in the DECREASE trials in the Netherlands, but 1 of the researchers was later fired for scientific misconduct. The original data could not be scrutinized by investigational committees, but when that study was left out of metaanalyses, β-blockers increased mortality. Another example is the fraudulent report from the United Kingdom associating measles vaccination with autism, which led to a decrease in vaccination rates and increase in measles incidence, with fatalities. The second major consequence of breaches of scientific integrity, in all fields, is a loss of public trust in science. Such a lowering of trust will make it easier for politicians who do not like certain research findings, as, for instance, on climate change, to discredit or ignore whatever does not match their personal view.

What steps can be taken to counteract the detrimental effect of predatory journals?

Arthur L. Caplan: The problem of what I have termed publication pollution is causing the climate of science to deteriorate. Institutions must be clear that they give no credit to work that appears in predatory journals. Scientific societies must track and rank bogus journals in their respective fields and publicize the bad ones. Funders must demand the return of funds for work that appears in bogus, fake predatory outlets.

Deborah J. Sweet: The best counterpoint here is education to help authors avoid submitting to these journals in the first place. I can understand that it is sometimes difficult to tell just from an e-mail or even a website which journals are conducting bona fide review and publishing processes and which are not. But, just as education can help us with identifying phishing e-mails or other types of scam, it can also help with avoiding predatory journals. A straightforward test is whether the journal comes from a known and reputable publisher. In addition, authors should look at whether the journal is listed in vetted indexing services such as Scopus, and look at the overall volume, frequency, and quality of papers already listed as being published. If they'd like to go an extra mile, they could e-mail one of the listed corresponding authors to ask about their publishing experience. There are of course legal recourses as well, but those are more in the purview of regulatory bodies and government institutions rather than individuals.

Peter Hornung: Awareness, demarcation, outlawing. The business model of the predatory publishers is based on ignorance and disinterest of many scientists and the unscrupulousness of some. The information provided by universities and university libraries on scientific publishing must be intensified. We need reliable black and white lists that make it possible to quickly check on the basis of comprehensible criteria. It is less a matter of developing a static definition of the term predatory publisher. It is more important to investigate and set red flags for each publisher: unrealistic deadlines for peer review, uncritical acceptance of new reviewers, exaggerated advertising promises, false or misleading information about the company's registered office, and acceptance of editors without them knowing about it. This makes it understandable for every scientist in a specific case why a certain publisher should be avoided.

At the same time, it is necessary to draw a clear line between serious science and any area of publishing that clearly, repeatedly, and fundamentally violates scientific principles. This demarcation has increasingly failed in recent years. This can be seen from the fact that supposedly serious scientists also publish with predatory publishers, that the names of important universities and research institutions appear in publications by such publishers, that such publications can be found in the catalogs of university libraries without the dubious place of publication being named, and that such publications are quoted from and are found as sources in doctoral theses.

The clear line of demarcation must be accompanied by an outlawing of those who do not respect this demarcation. It must be clear to those scientists that they are harming themselves, their university, and science in general. “Nobody will notice,” was the attitude of numerous scientists so far. Or, “I didn't know that at all.” If clear information is available and anyone can check whether an offer is serious without much effort, that should no longer be an excuse.

Frits R. Rosendaal: Predatory journals are those that ask a fee but do not provide the service, i.e., good peer review, copyediting, typesetting. They are there to rip off authors and are a growing phenomenon, mainly because of the possibility of online publishing, which requires much fewer investments than printing and posting. There is evidence that some authors purposefully publish in these journals to create a quantitatively impressive publication list, so as to mislead promotion committees. Therefore, maybe the correct analogy is not of predator and prey, but one of symbiosis. In this case, looking at impact factors may be helpful. Further than that, predatory journals are no threat, at least not to scientific integrity. One might even argue that serious print journals that are owned by large publishers or scientific organizations, and that make enormous profits per year through university library subscriptions, are more predatory.

What are the responsibilities of editors and publishers in assuring the integrity of the publishing enterprise?

Scott Moore: Editors and publishers are in a unique position in the research enterprise. Research funded by federal grants is generally intended to be disseminated so that researchers can continue to build on each other's work. Editors and publishers who encounter research misconduct allegations should take them seriously and report those allegations to the funding agencies when a federal grant is acknowledged for support. There are processes and procedures that ORI and our federal partners follow to ensure fairness in resolving allegations. The process is not as instantaneous as some would like, but the care taken to respect privacy interests of those involved and to provide a thorough and fair review is important.

Arthur L. Caplan: Editors and publishers must insist on and pay for quality peer review. They must be on the alert for retracted work. They must seek to provide statistical help to editors as needed. The profit margins of publishers are large enough to promote more training, assistance, and oversight of publishing.

Deborah J. Sweet: As journal publishers, we are responsible for overseeing 1 stage of the overall research cycle, right at the end. Therefore, while a few elements of the process are in our control, many are not. We also have somewhat limited information available to us and, therefore, have to rely on trust to a large extent. Nevertheless, we do have a substantial responsibility to be as rigorous as possible over the aspects that we oversee, and to do what we can to help our authors and reviewers participate ethically and rigorously as well. There are many ways in which we can do that, and organizations such as the International Committee of Medical Journal Editors and the Committee on Publication Ethics give valuable guidance in this regard. Almost all journals have guidelines for authors, editors, and reviewers that spell out their responsibilities and the types of conflict that they are expected to avoid, and we can make adjustments or take corrective action when we learn that there is a problem. Most journals also have a system for asking authors to declare any conflicts of interest with their paper. Software packages can help journals check for plagiarism and are effective at doing so. A number of journals, including ours, now also have specific checks of figures to look for evidence of inappropriate processing, and I look forward to the day when that process is quick and automated. Information provision is another opportunity, and in that vein, 1 of our major recent initiatives related to overall integrity is STAR Methods (https://www.cell.com/star-methods), in which we introduced detailed and structured reporting to help ensure clear communication of research methods and thereby promote reproducibility. More broadly, the increasing trend towards transparency of underlying data, while sometimes burdensome, can help insure against any misrepresentation, whether inadvertent or intentional, and journals that allow, encourage, or even require such deposition can help promote integrity. At Cell Press, as we expand our remit beyond life science and into physical sciences, we are learning that expectations and standards can vary substantially between fields, but we do view it as part of our responsibility to continually examine our standards and push to improve in line with community expectations of best practice.

Peter Hornung: Their task is to stand up for quality and principles. It is the responsibility of the publishers to select the editors in such a way as to ensure that the content of the journals is managed seriously. Ultimately, they are also responsible for what the scientific editors do. Predatory publishers are usually different: The Chief Executive Officer of Omics International, Dr. Srinababu Gedela, recently said in an interview during our investigation that his company only offers a platform for publications. The editors, in turn, are responsible for the scientific quality of the journals with their names on the masthead. When they discover that a publisher is pursuing a business policy that questions the scientific quality of their journal, they must oppose it and, ultimately, withdraw as an editor. This also applies if they can guarantee the quality of their own journal, but the publisher grossly violates scientific standards in other cases. After all, such behavior will ultimately harm all scientists whose names are mentioned in connection with such a publisher.

Frits R. Rosendaal: Editors and publishers can only do so much, and the main responsibility lies with researchers and their institutions. Still, journals can use plagiarism software, ask for complete and unmodified blots of laboratory assays, and, first and foremost, use seasoned reviewers. Skewed articles are probably more insidious and widespread than outright fraud, so particularly in the medical field where financial stakes are high and risks potentially life-threatening, policies on conflicts of interest could be intensified, e.g., by not publishing reviews by authors with financial ties with commercial entities. But journal editors are not police detectives who can investigate, so largely publish by trust in researchers. Researchers have the task to earn this trust.

Academic promotion committees' and funding agencies' emphasis on journals' impact factor (IF) has been questioned. In this environment of journals' proliferation and publication of questionable results, do you see the role of IF subsiding or becoming more prominent?

Arthur L. Caplan: IF is 1 element for promotion and funding, but only an element, not the last word. Referees involved in promotion must take the time to read and evaluate a sample of papers. Time must be given and recognition accorded to serious peer review.

Deborah J. Sweet: The IF is only 1 of many measures of the overall quality of a journal. There are alternatives to the impact factor (e.g., CiteScore, Altmetric Score) that when considered together can give a broader perspective on a journal's standing, but every metric has inherent limitations. The IF is easy to understand, widely quoted, and, perhaps most importantly from the perspective of application, available by association as soon as a paper is published. So, it is perhaps not entirely surprising that hiring and funding committees seem to have continued to use it as a benchmark despite widespread recognition of the limitations of doing so. In general, the IF tends to correlate with other journal indices, and it is a measure of the mean citation of previously published work, but overreliance on the IF or any journal-based metric to judge the value of an individual paper or a researcher's contribution to the literature is not appropriate. Ultimately, there is really no replacement for actually reading a paper and formulating an opinion. I haven't noticed much change in the role of the IF related to increasing numbers of journals. I find it hard to see a really substantial change happening in this regard until someone comes up with a different assessment method that is equally easy to use and gives a better outcome. But, whether that would relate to journal articles per se or some other way of measuring contribution is still up for debate.

Peter Hornung: Not fundamentally changing at all because predatory publishers have usually already been recognized as such by the relevant organizations, do not have a recognized IF, and are not listed in the corresponding directories such as the Web of Science, Directory of Open Access Journals, or PubMed. Predatory publishers therefore try to advertise themselves with their own IFs, often calculated on an unclear basis or even fictitiously invented. Scientists should know this to avoid being deceived. Basically, the IF is nothing more than an auxiliary construction to capture the quality of a journal. Nevertheless, it has an important function in distinguishing the serious from the dubious area.

Frits R. Rosendaal: Impact factors, number of publications, author position or citation counts are all proxies for scientific quality. Unfortunately, as Goodhart's law states, whenever a measure becomes a target, it stops being a measure. So, promotion committees should try and really evaluate quality, which will cost more scrutiny and time than just tallying quantitative measures. Given that critiques on the quantitative measures are increasing, I am somewhat optimistic.

Are currently established conflicts of interest (COIs) criteria for editors, authors, and reviewers adequate to safeguard the system, or do more stringent requirements need to be added?

Deborah J. Sweet: The standards and expectations here seem to me to vary between journals and between fields. In general, though, we basically operate on an “honor system” where we expect editors to avoid handling papers where they have a conflict, reviewers to decline to review when they do, and authors to be open and transparent about any conflicts that do exist. My general impression is that the criteria and expectations at reputable journals are reasonable for their respective fields and strike a good balance, and that when problems arise they come because these expectations were not met in 1 way or another. Nevertheless, we probably do have room to improve in terms of communication and enforcement. At Cell Press, we have clear guidelines for authors and reviewers, and we train our editors to err on the side of caution if a potential conflict does arise. We also recently enhanced our Declaration of Interests policy for authors to make our expectations clearer and the information more transparent, and we are considering further changes to improve it even more.

Arthur L. Caplan: Current COI standards are confusing, inconsistent, and nearly worthless. An upgrade to the primitive state of COI disclosure is not only necessary but long overdue. Transparency in all facets of research and scholarship cannot be achieved by listing only an entity that one works with. Without substantive information about the nature of that work, simple disclosure is inadequate. Standardized COI templates ought to be created. These should be electronic and readily available to all who care to view them. Researchers should create and maintain their own detailed statements and make them publicly available online. Researchers should use the space afforded by online disclosure to explain what they are doing and why. Electronic COI disclosure should be linked to an ORCID number, eligibility to receive public funding, and to filing trial data with sites such as the NIH's clinicaltrials.gov. COI needs to become standardized, electronic, explanatory, and mandatory.

Frits R. Rosendaal: Particularly in medical research, COIs are a major concern. The current idea is that disclosure is sufficient. Whereas there have been recent cases in which financial relations were not disclosed, the more relevant question is whether, when we believe that conflicts may bias research outcomes, disclosure will unbias. This seems unlikely. However, as long as many physicians enter financial relationships with commercial entities, and also much of the research with drugs is financed by these companies, it is unclear how this bias could be removed. Editors should strive that opinion pieces are only written by individuals without conflicts.

• Received for publication October 25, 2018.
• Accepted for publication November 29, 2018.

• © 2018 American Association for Clinical Chemistry