Journal Peer Review: Tips for Being an Effective Reviewer

Catherine Cocks's picture

Guest post by Michael Chibnik, Emeritus Professor of Anthropology, University of Iowa

From 2012 to 2016, I was editor-in-chief of American Anthropologist (AA), the flagship publication of the American Anthropological Association. We required at least three peer reviews for each manuscript submitted for consideration as a research article. Editors sometimes have trouble finding people willing to comment on manuscripts. Such reviewing may be regarded as volunteer unpaid labor that takes time away from more important work. I nonetheless would encourage scholars to accept when possible requests from editors to review. In most disciplines, researchers depend heavily on academic journals to learn about recent developments in their areas of interest. Editors need peer reviews when making decisions about which articles will appear in their journals. Providing comments on manuscripts need not be an onerous task. I rarely take more than five or six hours to read a submission and write a review.

Every manuscript I sent out had some merit; submissions that were clearly inappropriate for the journal were rejected without peer review. Although commentators were given general guidelines for what was expected in a review, they were able to write whatever they wanted. In my four years of editing, I read several thousand reviews. While most reviews were helpful, some were more useful than others. Here are three suggestions about peer reviewing, based on my experience:

1. Pay attention to any instructions or guidelines. These guidelines indicate what an editor is looking for. While I would have been glad to respond to questions about the detailed instructions we sent to reviewers, I never got any. I was often surprised when reviewers totally ignored my guidelines and chose to comment at length about minor issues such as small geographical variations in cultural practices. 

2. Reviews should be neither too short nor too long. Perhaps the most useless reviews I received were either brief expressions of approval (“well done” was the entirety of one) or abrupt dismissals. Such comments suggested that reviewers did not take their task seriously. I ignored these kinds of reviews when making manuscript decisions and sometimes lost time because of the need to find other, more thoughtful, reviewers. Although less problematic, some reviews were too lengthy and must have involved extraordinary amounts of work. A page or two of comments ordinarily sufficed for American Anthropologist.

The academic folklore is full of stories of reviewers misunderstanding manuscripts and providing destructive, nasty comments...these comments told me more about the reviewer than about the manuscript.

3. Reviews should be polite and constructive. The academic folklore is full of stories of reviewers misunderstanding manuscripts and providing destructive, nasty comments. While the great majority of reviews submitted to AA were constructive, a few were gratuitously mean. One reviewer, for example, called the paper he was sent “a disaster,” characterized the theoretical section as “abysmal,” and went on in this unpleasant vein for three single-spaced pages with no paragraphs. I thought that these comments told me more about the reviewer than about the manuscript. After some thought, I sent this review to the author with a note stating that I disliked the report’s tone and did not take it into account in my decision making. Many journals, including AA, have separate sections in their review forms for comments to the author and comments to the editor. If a submission is really bad, the place to say so is in the comments to the editor.

There are three general types of comments that often should be included in peer reviews. The most obvious concern the quality of the manuscript. Almost every editor wants to know about a submission’s significance and originality and whether the researcher’s methods are reasonable. Reviewers should in addition pay attention to the fit between theoretical and descriptive sections of a manuscript. Authors too often cite theory that is only marginally relevant to the research findings they report. Editors, of course, are also interested if relevant previous work is cited. However, some reviewers place too much emphasis on this in their recommendations about suitability for publication. Insufficient attention to previous studies is a shortcoming that can often be remedied in revisions.

Secondly, reviewers should, when necessary, comment on the structure and prose of manuscripts. Such comments are an especially constructive way for peer reviewers to help authors, who often have difficulty recognizing organizational shortcomings and writing problems in their manuscripts. In their introductions, authors frequently have trouble previewing arguments and providing concise descriptions of the evidence underlying their conclusions. Some sections of a manuscript might be too lengthy and others too brief. Material may be presented in a confusing order. Perhaps more than most editors, I was also concerned about the quality of writing in a manuscript. I am not alone in observing that academic prose often includes opaque jargon and convoluted sentences. I was sometimes astonished when reviewers praised the writing in manuscripts that I found very difficult to read. 

Finally, a manuscript may not be a good fit for a journal. In the case of AA, such submissions could be overly descriptive or overly technical. I was grateful when reviewers suggested other journals where a manuscript might be more suitable.

Authors usually appreciate constructive feedback about the strengths and shortcomings of their scholarly work. When peer reviewers provide such comments, they are making an essential—if often undervalued—contribution to scholarship.

Although peer reviews were important influences on my manuscript decisions, it was not always easy to assess their diverse and sometimes contradictory comments. Furthermore, decisions were also influenced by my own reading of manuscripts, which usually differed in some respect from those of the reviewers. The great majority of articles published in AA received initial revise and resubmit decisions. In my letters to authors receiving such decisions, I backed up many of my suggestions for revisions with direct quotes from different reviewers. I often advised authors to pay particular attention to certain comments of reviewers and sometimes suggested that they ignore other comments. When reviewers provided lengthy comments that were not central to my evaluation of a manuscript (usually about ethnographic matters I was unfamiliar with), I would recommend that authors “look at reviewer X’s comments and make whatever changes you think appropriate.” 

Journals vary in the types and amounts of feedback they give to manuscript reviewers. Some let reviewers know decisions about publication and provide copies of the comments of all reviewers. Others let reviewers know of decisions, but do not provide copies of reviews. Many give reviewers no feedback at all. When I became AA editor, reviewers received only a form letter thanking them for their comments. I thought this was insufficient, given the service provided by reviewers. After consulting with our editorial board and other anthropologists, I began telling reviewers about manuscript decisions and giving them access to the comments of all reviewers. (When people were asked to review manuscripts, they were informed of this policy.)

AA uses a system of double blinding in which authors and reviewers in theory do not know one another’s names. Many journals do not use double blinding. When I edited Anthropology of Work Review, for instance, we used single blinding in which reviewers knew the names of authors, but authors did not know the names of reviewers. Single blinding and nonblinding are the most common practices in journals in the natural sciences. I would guess that these differences in blinding practices have some influence on the content of reviews, but am not sure about this.

Peer reviewers should keep in mind that their comments are aimed at both editors and authors. Editors are concerned both about making decisions about whether or not to accept manuscripts and about ways to provide concise advice to authors about necessary revisions prior to possible publication. Authors usually appreciate constructive feedback about the strengths and shortcomings of their scholarly work. When peer reviewers provide such comments, they are making an essential—if often undervalued—contribution to scholarship.

Michael Chibnik is Emeritus Professor of Anthropology at the University of Iowa and the author of Scholarship, Money, and Prose Behind the Scenes at an Academic Journal (University of Pennsylvania Press, 2020).

Have something to say on this topic? Reply to this post or email the Elephant about writing for us. We welcome submissions from stakeholders on all sides of scholarly publishing.