A case study of peer review enabling academic abuse
Feb 23, 2024
Science was a beautiful thing before people got involved.
As a child, the son of a molecular biologist, I envisioned Science as a magical community of scholars all committed to truth, a Vulcan-esque species of minds for whom evidence and logic reigned supreme over the malign influences of emotion and ego. In pursuit of this ideal, I strove for excellence, graduating summa cum laude with degrees in mathematics and biology, finishing a PhD from Princeton in 4.5 years, and waiting to publish results until I could overcome my own attempts at objective self-criticism. I would wait to speak until I felt I truly had something novel and significant to say.
When I spoke up in the form of papers, however, I began to realize that Science is nowhere close to the idealized system I dreamt about. My first paper, written as an undergrad, sat on someone’s desk for 6 months during which time other authors submitted & published a paper which was, 6 months after my submission, used to reject my manuscript and claim it is not novel. Years later, I met the people who were my peer reviewers, they realized whose student I was, and, perhaps without realizing the hurt they would cause, they let it slip that they would’ve accepted my paper if they knew I was the student of such a prestigious advisor.
A similar experience happened to a friend of mine who submitted a paper, only to find the editor overseeing his paper published a replica of his paper during peer-review, complete with the same maps my friend included in his manuscript. The editor then rejected the paper, again about 6 months later, claiming the paper was not novel. My friend, indignant, wrote the journal asking why the editor didn’t recuse himself given he was working on such a similar topic at the time of handling the paper, but the editor was too powerful, he came down with the hammer against my friend, saying how dare he accuse The Editor of misconduct! The Editor remained the editor, a threatening symbol of power and abuse visible to all who knew of this episode, a lingering flame of injustice waiting to burn anyone else who came close.
There are countless other battles, from editorial desk rejections claiming subjectively and without cause that a very important paper is “not impactful” to the notorious “Reviewer 2”, a name academics reserve for reviewers who blockade a good paper with unhelpful, wrong, or sometimes even hurtful criticism that has nothing to do with the paper or the ideas presented therein.
The frictions of peer review are even worse for people who are truly different, and I count myself as one of the truly different scientists not out of flattery but out of necessity to contextualize the unique pains of my academic journey. Most biologists are not fluent in math, and most mathematicians are not fluent in biology. I happen to be fluent in both. I was the first graduate from the first class of Princeton’s interdisciplinary Quantitative and Computational Biology program, where I joined people in the lab developing novel assays, in the field surveying birds and vegetation in the Amazon, and at chalkboards hashing out complex mathematics.
Any academic paper I write sounds different because I fluently speak two disciplinary dialects, and I have an accent with than just these two topics of my formal education because I’m interested in many things (e.g. I worked with a hedge fund for 7 years doing quantitative finance – that leaves a lisp of forecasting and risk-adjusted returns). In peer review, I’ve been told my work is “too bold”, “too mathematical”, “too technical”, “not impactful” and more. I’ve been told that the mathematical heart and soul of my mathematical biology work should be moved to a supplement by a biologist unfamiliar with the math.
After years of suffering through peer-review whose main effect was to diminish the authenticity of my work, work that I published only after it withstood my unbearably cruel self-criticism, I’ve come to believe that peer-review is not my peer. If you are truly independent, authentic, and creative, you are doomed to be perpetually misunderstood by people who are not these things, and those people will be your peer-reviewers. If you find yourself on the frontiers of knowledge, expressing ideas that take time and careful thought to evaluate, you are liable to be the target of cursory reviews and constant misunderstandings. After years of being unable to publish your academic ideals without significant friction and fights, one becomes susceptible to a learned helplessness, a chronic state of discouragement and belief that you cannot achieve your goals within the current system.
The public is not aware of the battles that take place behind peer-review. They absolutely need to be informed, however, that peer-review is broken, that most published findings are false, that inconvenient truths can be suppressed and correct paradigms obstructed thanks to the power imbalances and the academic intolerance empowered by peer-review. These inefficiencies, what I call “socioscientific inefficiencies”, have delayed our society’s ability to adapt to a pandemic by delaying the progression of knowledge about SARS-CoV-2 epidemiology, the efficacy of non-pharmaceutical interventions, and more. To make matters more dire, these same inefficiencies, power imbalances, and back-door abuses of editorial and committee power may have enabled scientists’ mismanagement of risk and caused the pandemic.
The public sees peer-reviewed papers like Proximal Origins, claiming a lab origin is “implausible”, but until recently they didn’t see the back-door acknowledgements by the authors that a lab origin was “so friggin likely”, or the role of Scientists For Science who lobbied for risky research in pressuring the authors, or the role of Jeremy Farrar at the Wellcome Trust ghostwriting the paper. The public saw the head of NIAID, who funded coronavirus work at the labs in Wuhan, present this paper without revealing his own role in prompting and leading the paper. The public saw New York Times articles stacking narratives on top of this hollow foundation, claiming that scientists have concluded the virus is zoonotic. These same authors proceeded to write a long list of similarly flawed papers, all of which received coverage in The Guardian, The New York Times, and beyond, all of which passed peer-review, yet all of which have been shown to be false. Bad statistics. Bugs in the code. Cherry-picking a single sample with a raccoon dog and SARS-CoV-2, overlooking the negative correlation between raccoon dogs and SARS-CoV-2.
On the other hand, you have a paper like ours, from independent, authentic, and honest scientists whose public and private views tell the same consistent story of belief in a laboratory origin.
You have the paper Valentin Bruttel, Tony VanDongen, and I wrote documenting the strange pattern of BsaI and BsmBI restriction sites in SARS-CoV-2, quantifying the anomalous nature of this pattern, and putting this pattern in context: it is an anomalous pattern among wild coronaviruses yet precisely the pattern we expect to see in a virus made in the lab. Over 100,000 people have read our full text, our findings have been replicated, our findings even predicted detailed methods in the DEFUSE grant proposing exactly what we theorized could create SARS-CoV-2 alongside order forms for the enzyme BsmBI, a prediction of scientific methods Nicholas Wade called “The Story of the Decade.”
Our paper has proven useful for forensic investigations… yet it is not peer-reviewed.
Do you want to know what happened to our paper?
Today, in the spirit of saying something novel that has withstood my own brutal self-criticism, I need to share with you all what happened with our paper in eLife. The short answer is that our paper was accepted, it was held up in eLife for the magical 6 months, and at the end of the 6 months the editor in chief was fired. Over 6 months after submission but a mere three days after the editor in chief was fired, the Co-Editor-In-Chief rejected our paper claiming they could not find enough “unbiased” reviewers.
Details below.
The eLife Peer Review Saga
Posting, Accepting Public Reveiws, and Submission
Our pre-print was posted October, 20th 2022. It was covered by The Economist with independent scientists finding our results compelling, but the scientists who authored the above Proximal Origin and Raccoon Dog studies contesting our work, leading the Economist to call our study “contested” and The Telegraph to cover our paper merely by discussing how “Scientists in a furious row”. Again, the furious scientists were the people with reputational risk in this matter, people like Eddie Holmes who was helping the Wuhan Institute of Virology characterize bat SARS-Coronaviruses at the time of emergence, a conflict of interest he failed to mention when writing his many now debunked papers claiming a lab origin was implausible.
We were accused of P-hacking. Nature Magazine’s Amy Maxmen claimed we did not have PhD’s in the relevant subject matter (Tony VanDongen is a professor of bioengineering at Duke, my PhD is on evolutionary statistics, etc.). Others, like Scientist for Science and gain of function research of concern lobbyist Freidmann Weber, criticized our work by misrepresenting scientific methods for making reverse genetics systems, claiming one can’t build a virus as we propose (clearly, that is refuted by DEFUSE proposing to build the virus as we propose).
October 24th, 2022, I addressed some of the acrimony with an open letter to the world clarifying our intentions and who we are. October 27th, 2022, one week and tens of thousands of views after posting our work, I summarized the debates and why our findings still stand.
We then looked around at journals and editorial boards to assess where we might get a fair review, especially in light of the acrimony and conflicts of interest that pervaded this field. During this time, eLife created a new model of peer-review where articles are accepted and published alongside their reviews, allowing scientists to see the deliberations and make up their own minds. Encouraged by this new model of peer review as a venue for open deliberation, we submitted our paper April 10th with a cover letter for the ages.
Editorial Rejection
A week after submission, we received an editorial rejection pointing us to a Wuhan Institute of Virology scientist’s work nominally rejecting our paper (but, in fact, corroborating our hypothesis), claims that RaTG13 could not be the backbone, and the cruel remark that the editor felt our work was not aimed at “a serious scientific attempt to address this question”. Welcome to peer-review.
We appealed the editorial decision, breaking down their arguments point-by-point. Our appeal was, admittedly, a bit long, but we felt this was necessary to communicate clearly and make all of our assumptions and logic transparent. You can read the rebuttal below, but the big picture is we had to effectively write a second paper addressing critiques, making many of the points we’ve already made, and ultimately adding novel ideas behind closed doors, without any credit (e.g. pointing out flaws in Dr. Wu’s paper – that, in and of itself, ought to be a paper citing Wu’s).
Acceptance!!!
After our appeal, we heard on June 7th that the editors accepted our paper in their new publication model of Reviewed Preprints!
We were elated!!! We believed this topic is of great importance and were very happy this could be discussed in open-access scientific literature, ensuring our paper can show up besides reviews from critics that we invited in our cover letter.
We were billed $2,000 to pay for the publication fees.
However, we don’t have NIH or NIAID funds that can fund this work. I would’ve had to use my company’s funds or we would’ve had to find other funds to pay for this publication. In due diligence, we couldn’t pay this invoice until we had a better sense of what the final product would look like and could ensure the final product served the interests of whoever is paying the bill.
Editor Fired, Paper Rejected
On October 24th, 2023, one year after submitting our open letter to the world, eLife’s Editor In Chief Dr. Michael Eisen was terminated for expressing pro-Palestinian views he has long held. The termination was a decision of the Board of Directors, which includes representatives from the Wellcome Trust. For context, you may recall that the Wellcome Trust funded the DEFUSE PI Peter Daszak’s work prior to the pandemic, and the leader of the Wellcome Trust, Jeremy Farrar, ghostwrote Proximal Origins and recruited a mix of gain of function lobbyists and other Wellcome Trust affiliates to author a letter to The Lancet calling lab origin theories “conspiracy theories”.
On October 27th, 2023, exactly one year after our iteration of public reviews, over 6 months from our April submission date, and a quick 3 days after Dr. Eisen’s unusual termination, we received a letter from the Co-Editor-In-Chief Dr. Detlef Weigel rejecting our paper.
The reasoning he provided included an unjustified personal concern that “the work is too speculative and that the major conclusions are not supported by robust phylogenetic analyses” (yet he does not provide those phylogenetic analyses to justify his claim; incidentally, I am an expert on robust phylogenetic analyses). Dr. Weigel continues: “Independent of this concern, we have – as you know – been trying to secure unbiased reviewers for a number of months now. Unfortunately, we have reached the stage where we need to accept that we cannot provide high-quality public reviews of your work.”
Recap: A Series of Failures
To recap, we posted our paper and were immediately attacked by people with a long history of publishing highly misleading work without acknowledgement of unethical ghostwriting or conflicts of interest in the matter of SARS-CoV-2 origins. The press coverage of our work did not focus on our work but instead gave unquestioned legitimacy to the controversy manufactured by conflicted scientists, and scientists who lobbied for the precise research hypothesized to have caused the pandemic.
Despite the acrimony, we remained polite and professional to the best of our ability. We listened to criticisms from people who accused us of fraud and included their points in both the discussion of our updated manuscript as well as the cover letter for our submission.
We submitted our paper to eLife and the editor initially rejected our work on disputable grounds. We disputed these grounds in an appeal, and our paper was accepted for peer-review under eLife’s new publication model promising to publish papers alongside peer-reviews. eLife sent us a $2,000 invoice, but we did not have the funds to pay for it unless we could ensure their presentation of our work was professional.
Suddenly, 6 months after our paper was submitted, months after it was accepted and we were invoiced, the Board of Directors, including representatives from the Wellcome Trust that funded researchers at the heart of research-related origin theories, fired Michael Eisen for expressing long-held views. Three days later, the new editor rejects our manuscript claiming, among other things, that they could not secure enough “unbiased” reviewers.
How did eLife determine reviewers were “biased”? Is this test of “bias” something they apply objectively across all papers, or was this applied selectively to our paper? In which way were reviewers “biased”? Is it possible the “biased” reviewers were those who published prior work claiming a lab origin is “implausible” without revealing the Wellcome Trust’s leader ghostwriting their work, their conflicts of interest working with the Wuhan Institute of Virology at the time of emergence, or possibly other co-authors of work claiming to find a zoonotic origin, work that has since been disproven?
Most importantly, why are the biases of reviewers used to reject our paper? Why are we being punished for the biases of others?
The peer-reviewers, editor in chief Detlef Weigel, and Board of Directors at eLife have empowered the prejudices of scientists and funders who have a lot to lose if our work is right.
Since rejection, our work was not only replicated, but it received remarkable corroboration by predicting contents of a grant written by people who had Wellcome Trust funding, who collaborated with our primary detractors to collect and document bat SARS coronaviruses in Wuhan, and who are closely connected colleagues of the primary detractors of our work.
Science Is Broken
In the magical universe of Platonic ideals, Science exists as a process to collectively produce evidence, develop theories, rigorously evaluate competing theories, and check for errors through the process of peer-review. The process of peer-review is so central to our ideals of science that many journalists will warn that a paper is “not peer-reviewed” and that only once it is peer-reviewed is the paper a part of science.
In reality, the sausage factory of peer review is tragically far from our scientific ideals. Scientists are incentivized to produce evidence corroborating their theories as such evidence advances their careers more than evidence rejecting their prior work. Journals are disincentivized from publishing null findings as they’re not as flashy as papers claiming to have found something, even if the thing they found was dumb luck, and journals additionally may have conflicts of interest if the funders or Board of Directors have ties to research subject to investigation or criticism. The process by which science becomes published requires exposing our work to anonymous peer-reviewers, many of whom may have adverse incentives including their reputational risk from a paper contradicting their prior work, or even the journal’s own reputational risk due to the activities of the journal’s editors, board members, or funders.
None of these risks of peer-review are new to scientists. In fact, these problems of peer review are precisely why Michael Eisen experimented with new publication models of eLife, and the new publication model at eLife was subjected to the ultimate test by our work. Ultimately, the Board of Directors overlooked their conflicts of interest, assumed control, appointed an editor of their choosing, and 3 days after Eisen’s termination, moments after their new editor’s appointment, our paper is rejected.
It’s hard to escape the cold, discouraging clutch of cynicism once you’ve stared inside the sausage factory of peer-review. It’s especially hard to overlook the ways in which scientists are abusing their institutional power and reputations to obstruct open deliberation on the possibility that these same scientists, their funders, and their colleagues, may have caused a pandemic. Given a paper with reproducible evidence consistent with a laboratory origin, so far our scientific system seems incapable of publishing it not because our paper is wrong (if it were, then that would be easy to show in a review) but because it is horribly inconvenient for many people with power.
Objectively, our paper has withstood an enormous volume of criticism, we have remained open to other possibilities, we have expressed our findings with humility aware of other possibilities, and our paper has even predicted precise methodological details written in a draft of a grant, the central grant to lab origin theory, information that we were not privy to at the time of our work.
I don’t need peer-review because I am my own worst critic, scientifically stricter yet emotionally kinder than our chief detractors, plus I don’t have conflicts of interest on this question. By my own self-criticism, our paper has proven useful. We have provided context for a forensic investigation of drafts of a grant, and it’s entirely possible that future evidence may further corroborate our finding. Had we not published our paper, the esoteric details in a 2018 proposal for “6 segment assembly” alongside cost estimates for BsmBI may have gone unnoticed in the 1,400 pages of emails and drafts. In light of our paper, however, such a finding carries significance, context, and meaning, but that meaning is not kind to those who have tried to stop our paper from being peer-reviewed.
We could still be wrong, and we continue to welcome diverse perspectives. That, it seems, is what makes us truly different from scientists who have acquired power in our modern academic system. When we uncovered something that implicated our colleagues, as Andersen and Holmes did in January of 2020, we did not cover it up. Instead, we have incurred immense costs, received unprecedented hostility, and been subject to historic alienation from an entire field of science for sharing what we truly believe in the form of a paper, with a github repo of annotated code, citations contextualizing our findings, all of which is reproducible.
If such honest, reproducible work cannot receive a peer-review, then these people are not our peers. If such discourse is discouraged, if scientists impose such harsh penalties on anybody who touches the inconvenient theory of a scientific accident, then perhaps our modern scientific system is not what we dreamed.
Your restraint as you write of your mistreatment is really admirable. May you find support in the future from an awakened community wanting your conscientious service. Hang in there! Some better form of peer review must surely be arrived at!