Publishing in peer-reviewed journals is a painstaking—and often painful—business. An editor may reject a submission within days, sometimes overnight. A manuscript might then wait weeks for reviewers to be found, only for them to interpret the work in conflicting ways, focus on the peripheral, or question decisions that appeared entirely reasonable. Some reports will be thoughtful and constructive; others cursory, contradictory, or difficult to use.
In one peer-review exercise I saw up close, one report consisted almost entirely of the assurance that the authors had “done a good job of the paper”. Another reviewer argued for amendments, working diligently through the paper’s questions, theory, method, and claims. In this case, one reviewer had offered a quick pat on the back; the other provided a diagnosis and a possible route forward.
This is the first difficulty of making sense of peer review. Reviews are far from objective. Products of an overloaded system, they may differ, sometimes pulling in diametrically opposite directions, for a variety of reasons—from cognitive biases to epistemic differences to egotism to ethical laxity to, yes, sheer laziness. So authors need to consider what the reviewers say, critically. They need to separate the verdict from the reasoning, then determine what kinds of problem the reviewers have identified.
The first skill in responding to peer review is therefore diagnosis. Is this a verdict or a reasoned argument? Is the problem local, structural, or foundational? Can it be clarified or revised, or should something be rethought or rebuilt? Should the author concede, clarify, or counter?
Recognising these distinctions does not make publication easy. Nor does it mean that reviewers are always correct. It does, however, make the process more legible. It helps authors decide what can be addressed or clarified, and what may need to be challenged.
That is where our discussion begins, with the practicalities of peer review. What are reviewers looking for? How should conflicting reports be read, and when should an author concede, clarify, or push back?
These practical questions lead to a more fundamental one. Peer review helps determine whether a paper meets a journal’s existing standards. Is that all there is to scholarship? Or might we—ought we—think of publication as a milestone on the way to a more idealistic destination?
We ought. We must. That’s the case I want to make.
Most reviewer criticism can be traced to three broad questions.While much in the academy has changed, the faults reviewers pick on have barely budged. In 1978, Charles Bonjean and Jan Hullum, writing in PS: Political Science & Politics, catalogued six hundred rejections and found the complaints already familiar. Half a century on, the diagnostic vocabulary remains stubbornly the same. Does the study matter? Can its claims be trusted? Does the paper deliver what it said it would?
The first concerns contribution. Reviewers may describe a paper as unsurprising, incremental, or insufficiently original. In one review process, the objection was expressed bluntly: “The manuscript do not advance what we know” [sic]; the findings were “generic and rather trivial”. The criticism was not that the topic, or even the study, was unimportant. It was that the authors had not shown what became newly understandable because of their inquiry.
A contribution does not have to be a dramatic discovery. It might provide new evidence, examine a familiar problem in a different context, challenge an accepted explanation, refine a concept, or show that an established pattern does not apply everywhere. But it should be sufficiently clear and considered; and the reader should not have to construct that contribution on the author’s behalf.
One reason may be that the focus is diffused. Papers often encounter difficulty because they attempt to do too much. A study may begin by examining news coverage, move towards social-media effects, and conclude with claims about public trust. Such linkages might be plausible, but the paper may not have the evidence or space to establish all three.
Reviewers then ask what the study is actually about. What is the key contribution and what might need to be removed so that it becomes visible?
Theoretical criticism often reflects the same uncertainty. A paper may contain a substantial literature review and still be theoretically weak. The problem then is not a shortage of relevant references, but that scholarship has been parachuted in without thinking through the ‘so what’; without being used to build an argument.
The literature we use should help define the problem, shape the research questions, and inform the analysis. Adding more citations will not resolve the problem if the relationship between those citations and the study remains unclear.
The second broad question that reviewers seek to be convinced on involves trust. Every research project involves choices: why this case, this sample, this period, this platform—and this method or approach. Authors may regard those choices as obvious, particularly when they know the subject well. Reviewers cannot be expected to make the same assumptions. “The sample is not justified” is hence a common criticism. It may not mean the sample is indefensible, just that the reasoning has not been made sufficiently visible.
Similarly, naming a method is not the same as explaining how it was operationalised. In one review for a prestigious journal, a reviewer began doing the arithmetic that the manuscript had left to its readers. A corpus became several national samples, then a much smaller group of texts for qualitative analysis, but the reason for each reduction was unclear. Who had coded the material? How had the interviewees been selected? How were their responses analysed? “This is all astonishingly vague,” the reviewer concluded. The problem was not a shortage of numbers—there was an abundance of that—but the absence of transparency about the corpus and its analysis. Without a rationale for those sampling decisions, and clear inclusion and exclusion criteria at each stage, the reader cannot assess the adequacy of the sample, the dependability of the analysis, or the claims being made on that basis.
The third question is whether the paper delivers what it promises. Research objectives, methods, findings, and conclusions need to fit together. A paper may ask a causal question but use a method capable only of identifying patterns. It may promise to examine audience responses but analyse only news content. It may present findings that are interesting but do not answer the questions established at the beginning.
Reviewers often describe this as a problem of alignment. The individual components may each be competent, but they do not yet form a convincing whole. In such cases, you might get a response that says, as one reviewer put it for a well-regarded journal, “There needs to be a better fit between the findings, methodology, and the objective of the study.”
Analysis is usually where this mis-alignment becomes most visible. Authors sometimes provide extensive description and assume that its significance in answering the research purpose is self-evident. A reviewer may then be left with data and the onerous task of piecing together what it all means. So the question is not simply, “What did you find?” It is also what the many ‘things’ you found mean, and what bigger story you can tell from them.
None of these judgements is mechanical. Reviewers differ in expertise, diligence, and interpretation. The same manuscript may receive one recommendation to accept and another to reject. A detailed report is not automatically correct because it is detailed, just as a positive report is not necessarily sound because it is welcome.
The recommendation and the reasoning should therefore be read separately. Accept, Revise, and Reject are verdicts. What is more important to the author is the argument underneath: what the reviewer has identified, how well the criticism is supported, how deep the problem runs and whether the report offers a credible route forward. That gives us a way of sorting criticism by depth:
In practice, these criticisms might be phrased differently. The constructive reviewers diagnose the problem(s) and offer a clear route towards revision. Others describe weaknesses that require the entire study to be rebuilt. And occasionally, a reviewer offers little more than a nod—or an extended growl.
When recommendations conflict—one reviewer suggesting minor revision, another demanding rejection—the useful question is not which report is kinder. It is where they agree, where they diverge, and what level of reasoning supports each diagnosis.
In one set of reports, both reviewers identified much the same weaknesses. In essence, an inadequately justified case, search terms that predisposed the findings, a thin conceptual foundation, and an opaque sample. They disagreed only on whether those problems could be repaired. One recommended rejection; the other major revision and re-review. When independent reviewers reach the same diagnosis by different routes, the author might want to pause and reflect.Agreement is not the normal condition. Peter Rothwell and Christopher Martyn, examining two neuroscience journals for Brain, found reviewers concurring on the verdict at about the rate chance would predict, and a later meta-analysis of 48 such studies found much the same. Such agreement is therefore atypical.
Another pair of reports split more starkly. One reviewer offered a series of specific, manageable changes and said, “I should like to see this paper published”. The other queried whether the interviewees belonged to the group the paper claimed to study, whether the findings were distinctive to that group, and whether the central concept had been established at all. That reviewer concluded that the paper “cannot be amended to bring it to a publishable standard”. The editor rejected it.
A harsh recommendation to a prestigious journal is usually taken at its word by the editors, unless it is plainly lazy. It tends to deserve the credence, because such recommendations often reach into the foundations of the study. A missing citation can be supplied. A section can be reorganised. But if the sample does not represent the population the paper claims to study, or if the central concept has not been established, revision may require new data, new analysis, or a different argument.
This is the distinction between difficulty and fixability. A long list of comments may still describe a repairable paper. A single objection may be terminal if it undermines the study’s premise.
Once the criticism has been diagnosed, the author must decide how to respond. The typical expectation is a revised manuscript, with changes highlighted, together with a response sheet. The basic moves available to an author are straightforward: concede, clarify, and counter (politely).
Concede when the reviewer has identified a genuine weakness. A direct response (“That is a fair point”), followed by the action taken, usually works. There is little value in defending a sentence, paragraph, or structural choice that can be improved without damaging the paper.
Clarify when the criticism rests on a misunderstanding. But clarification alone may not be enough. If a reviewer misunderstood the paper, something in the text probably allowed that misunderstanding. Explain what was intended, then revise the manuscript so that the next reader does not encounter the same problem.
Counter when there are defensible scholarly or methodological reasons not to follow the suggestion. Be careful, though. A counterargument should be grounded in evidence, principle, or the limits of the study, not in the author’s attachment to the original version. It should also be framed politely—the last thing you want is to pick an unintentional fight, particularly with a growly reviewer—and state clearly whether any revision has been made in response to the underlying concern.
How might that look? Consider the approach of one of my wonderfully diplomatic colleagues, who has perfected the art of disarming reviewers. In one contested revision, his response sheet was less an exercise in defence than an intellectual dialogue. Where a referee identified a genuine blind spot, he conceded without fuss and revised. Where he disagreed, he politely noted he “begged to differ”, cited the relevant literature to hold his ground, and then—crucially—added a clarifying note to the manuscript anyway, recognising that if an expert had misunderstood him, the text had perhaps allowed it. Reading the exchange, one gets the impression of a scholar engaged in an enjoyable, confident conversation with his peers.
The way the response is framed is important, too. My colleague numbered every comment, aligned it to the action, and marked up the changes in the revision. An annotated manuscript allows the editor and reviewers to verify the changes quickly, soothing whatever nerves you might have unwittingly frayed, and it signals that the author has treated the process seriously.
We could, then, think of a useful response as having four parts:
Then there’s what happens when authors choose to treat the review as an argument to win. In one memorable revise-and-resubmit, the authors used the response sheet largely to defend their initial methodological choices. Rather than addressing the referee’s concerns, they cited—somewhat confrontationally—the other reviewer’s praise as a character witness for their paper. The reviewer’s second report begins courteously enough: “I see that you disagree with most of my suggestions and defend your initial choices rather than make adjustments to the paper.” The reviewer then dismantles the rebuttal—the authors having made it rather easy in their obstinacy—point by point, and a paper that could have been saved becomes a hard reject.
The lesson here is not that authors must appease a reviewer. But disagreement carries a burden of proof. A reviewer’s kind words are not evidence, and they cannot serve as a character witness for your paper. Nor do token, performative adjustments help—and it is rarely wise to strike a confrontational tone with an irate reviewer cloaked in anonymity.
Reviewers, of course, are not always careful and constructive—nor, for that matter, correct.
A short review is not necessarily a poor one. A reviewer may identify a decisive problem in a few sentences. Nor is a long review necessarily authoritative. Detail can reveal expertise and diligence, but it can also mask misunderstanding, bias, or a demand for a different paper.
Some reports offer almost nothing. The approving review discussed at the beginning consisted only of praise and good wishes. It gave the editor no reasoning to assess and the authors no guidance they could use, so the editor called in a third reviewer. Positive reviews can be lazy too.
Other reviews are brusque or poorly expressed but contain a valid objection. In one rejected submission, a terse and unpolished report nevertheless raised a substantive question: “Sample: Sample is not justified.” A second reviewer reached the same concern through a much more detailed examination of how the outlets had been selected. The difference in register did not erase the diagnosis.
Authors therefore have to read past tone. What, precisely, is being claimed? Is the criticism supported? Does it identify a weakness that another reader could reasonably encounter? Is there an actionable point beneath the frustration, the profusion of exclamation and question marks that pockmark the review?
Sometimes the answer will be no. A report may misread the paper, make incompatible demands, or provide too little explanation to guide revision. In such cases, the author has the option of a considered rebuttal. If a request is genuinely impossible or contradictory, that is something to take up with the editor.
But all of this, the diagnosis and the diplomacy alike, is about surviving the process. It leaves unanswered the question we began with, whether publishability is all there is to scholarship.
Peer review, as these cases demonstrate, is fragile, fallible—a deeply human business. It lends an aura of scientific rigourThis aura is not as old as we tend to think. Nature did not require external refereeing of everything it published until 1973. The historian Melinda Baldwin has traced the term “peer review” itself not to journals but to Cold War America’s grant committees and medical review boards—JAMA would not use it of journal refereeing until 1981. When Congress turned on federally funded science in the 1970s, scientists reached for the phrase, which implicitly cast anyone without a scientific background as unqualified to judge the work, and argued that taking funding decisions out of expert hands would corrupt science itself. Few came away from the 1975 hearings satisfied, but Congress got no oversight of the grants. What we take for a timeless guarantor of scientific quality is a rhetorical settlement barely 50 years old, in which an optional bureaucratic process was recast as the guarantee of scientific legitimacy—though informal refereeing dates to 18th- and 19th-century scholarly communities. to what is fundamentally an uneven, subjective transaction. In Richard Smith’s memorable words, it is “something of a lottery”, which, “in addition to being poor at detecting gross defects and almost useless for detecting fraud”, is “prone to bias, and easily abused”.Smith had the evidence. Under his editorship, the BMJ planted deliberate errors into accepted papers and sent them out unannounced to reviewers. In the first trial, researchers introduced eight deliberate weaknesses: the average reviewer spotted only two; about one in six caught none. Some 10 years later, when a similar experiment was repeated at an even larger scale across hundreds of reviewers, the outcome was virtually identical—reviewers still missed most of the major flaws, and even formal training did little to improve the results.
How, then, might we navigate all this without letting it define our scholarship? Might we take refuge in the philosophy of Doctor of Philosophy, and look for solace in its genealogy? Tracing the title, we can find at least three distinct points, each leaving behind a different set of values and, together, if we choose to see it, offering an alternative to the metric-driven culture we now take for granted.
Philosophia meant, literally, the love of wisdom. For the ancient Greeks, philosophy was not principally a body of knowledge to be mastered, but an inquiry into how one ought to live. When Socrates told the Athenian court that the unexamined life was not worth living, he placed questioning at the centre of a life consciously lived. Aristotle would later give that life a civic setting. Human flourishing belonged within the polis; the purpose of political community was not merely to make life possible, but to make the good life possible.
For a scholar, this poses a fundamental question. Is this work worth the time I am about to pour into it? How might it, directly or indirectly, help the world? Research consumes us, directs our attention towards some problems rather than others, and eventually places something out into our world. Choosing a question is therefore choosing what deserves our attention and what, through that attention, we hope to contribute beyond ourselves.
Later, as medieval universities began to take shape, the idea of knowledge passing from one generation to the next acquired an institutional form. The early doctorate, licentia docendi, was literally the licence to teach. Scholarship, at this point in time, was understood as the transmission of mastery from one generation to the next. It was an act of inheritance. In 1159, the medieval scholar John of Salisbury recorded a remark in Metalogicon from Bernard of Chartres: we are like “dwarfs perched on the shoulders of giants”.The remark was not his own. In Metalogicon (Book III, Chapter 4), John of Salisbury attributes the image to his teacher’s teacher, Bernard of Chartres—making the metaphor an inheritance transmitted by the exact mechanism it describes. Robert K. Merton later traced its lineage from Bernard through Isaac Newton’s 1675 letter to Robert Hooke and into modern thought in On the Shoulders of Giants: A Shandean Postscript (1965), recursively performing the very cumulative scholarship the aphorism celebrates. We see more and further than our predecessors, Bernard observed, not through any sharpness of our own sight or stature of our bodies, but because we are lifted up and borne aloft on their gigantic mass. It is a wonderful image that captures the passing of knowledge.
We could, hence, think of the peer-reviewed paper, too, as an act of inheritance. Every piece of research enters a conversation that began before us. The concepts, methods, archives, and problems we work with have been handed down through the sustained labour of others. Citation, at its best, is not a ritual of academic compliance, but an acknowledgement of that intellectual debt. Yet an inheritance is not only received; it is sorted. Not everything handed down should be repeated uncritically. Some traditions preserve vital wisdom; others entrench omissions and prejudices and errors. The question then is not simply, What can I add to the growing pile, but what ought I to carry forward—and what, in carrying it, ought I to preserve, correct, or renew?
In the early nineteenth century, Wilhelm von Humboldt articulated another ideal for the modern research university: Wissenschaft, the sustained pursuit of knowledge for its own sake, coupled with Bildung, the personal and intellectual formation of the scholar. The university, in his view, should treat science (and knowledge) as “a problem which has not yet been fully resolved”, to be pursued in a state of constant research. The ethos of an institution where ‘new’ knowledge is pursued and shared is worth preserving. And yet the university we actually work in has been threatening it for a long time.
As early as 1942, in The Academic Man, the sociologist Logan Wilson noted how institutional pressures were already shifting the scholarly climate. “The prevailing pragmatism forced upon the academic group,” he wrote, “is that one must write something and get it into print. Situational imperatives dictate a ‘publish or perish’ credo within the ranks.”The phrase ‘publish or perish’ is often credited to Logan Wilson, but as Jeffrey Aronson traces in the BMJ, its pedigree is considerably older. Variations were circulating among geographers as early as 1904, and by 1938 Isaiah Bowman cited it as an established grievance. Decades later, Eugene Garfield tried to track down its first author without success—proving, perhaps, that some academic anxieties are timeless. Over the eight decades since, that situational imperative has hardened into a sprawling audit culture of journal rankings, impact factors, and output counts. A reasonable expectation—that scholars should contribute to the collective pool of knowledge—has been converted into an instruction to produce countable units.
Today, that machinery is buckling under its own weight. It was broken even when it seemed unbroken, and is now broken beyond repair. That it still yields value—a sound diagnosis here, a rescued paper there—is no more proof of health than an A&E that still treats patients is proof of a functioning NHS. Editorial desks are flooded. Journals, old and young, prestigious and peripheral, scramble for reviewers who have neither the time nor the inclination. The reports that trickle back are increasingly performative or perfunctory, and what began as a community testing ideas has turned into an exhausted, metric-driven industry.
This, I think, is where we can draw sustenance from that neglected word of the traditional doctoral title: philosophy. In borrowing from the traditions it has traversed—the Greeks asking how to live; the medieval masters teaching us to inherit, to stand on the shoulders of those before us; Humboldt insisting on inquiry—we might ask what mere publishability cannot tell us. Is this inquiry worth the months and years we pour into it? Am I genuinely contributing something that clarifies the world, or am I merely adding to the noise, to the research pollution? What am I carrying forward from those who came before me, and what should I have the courage to question or leave behind? And what good does the work place back into the community?
A journal may test whether an argument holds together, or we may choose to bypass the traditional machinery altogether—publishing open-access, on independent platforms, in repositories, or in formats of our own making.This movement is acquiring more popularity. Physicists have been posting to arXiv since 1991, and mathematicians like Tim Gowers run overlay periodicals that evaluate papers without bothering to host them. In 2023, the journal eLife abandoned the customary accept-or-reject verdict altogether, publishing manuscripts alongside the unvarnished referee reports and leaving readers to decide for themselves. The scholarly community largely shrugged: over a hundred institutions and funders confirmed they would keep counting eLife papers. The metric-counters did not. Clarivate removed the journal from the index that generates the Impact Factor, on the ground that it had decoupled publication from validation by peer review, and eLife chose to lose the number rather than the model. But wherever the work appears, the judgement on what to inquire into remains ours.
To publish with purpose requires three things of us. The first is the craft of research: doing the work rigorously, testing our claims against evidence, and making our arguments legible and useful. The second is inheritance: knowing what we have been handed, and deciding what of it to carry forward and what to correct or leave behind. The third is far more demanding: deciding for ourselves what is worth our attention, and refusing to let an exhausted, metric-driven industry decide what we ought to do, think.
Learn the rules of engagement if you choose to enter the peer-review tunnel, or build your own avenues if you do not—knowing full well that such blasphemy will attract its share of penance from the metric-counters. Make the work as rigorous and honest as it can be. But above all, choose carefully what is worth giving your attention—part of your life—to. Is it genuine inquiry, or merely the performance of it, to appease the cold gods of quantity?