The Refereeing Process in Economics Journals

The peer-review process is an essential part of academic publishing. We use it in the hope of ensuring the honesty, novelty, importance, and timeliness of published research. The selection of (usually anonymous) referees by a representative of the journal to which a research paper has been submitted for consideration, and the preparation of the reports/reviews by those referees, are key steps in the overall process of the dissemination of research results.

There are several different "models" when it comes to the refereeing, or peer-review process. Some of these have been described and compared recently, and in detail, here. It's also interesting to note that peer-reviewing is actually a relatively recent phenomenon in most academic disciplines.

There's no doubt that a well-crafted referee's report is a blessing - to both the recipient author and the handling Editor/Associate Editor/Editorial Board member who's looking to that report for an informed basis for making an editorial decision.

Unfortunately, such reports are not necessarily the norm in Economics/Econometrics - more on this below!

I know this is so, all too well - not only from the times when, as an author, I've been "on the receiving end" of some decidedly unhelpful reports; but also (and much more importantly) from my experiences on the other side of the fence, as a "handling editor" for a quite a number of economics, econometrics, and statistics journals.

Some would say that the academic publishing process is a bit of a crap-shoot. At times, I think that there's some truth to that. However, there's a great deal that both authors and referees can do to make the exercise more rational. 

Over the past 40 years or so I've read one heck of a lot of referees' reports. Right now, I read several every day, on the "editor" side of the process. These reports range from the sublime to the ridiculous, as the saying goes. Most are adequate.

Many are thoughtful and constructive, and they clearly reflect the considerable amount of time and effort that the referee has contributed to this (usually) pro bono task. They're vitally helpful to me and to the author(s) - regardless of the publication decision that's reached.

Some, unfortunately, are at the other end of the scale - for various reasons. Maybe the referee isn't as familiar with the paper's topic as the editor thought (s)he was. Or perhaps the referee was too busy to do a decent job, and really should have declined to take on the task at the outset.

And then there the occasional ones who have no feelings at all for the author(s). One of the worst examples of this that I encountered was during the period that I was an Associate Editor for Journal of Econometrics. A (well-known) referee provided an informed, but damning, report on a particular submission. The comments for the author(s), concluded with the suggestion that "...this dog should be put down!"

And so this brings me to the point of this post. What makes for a good referee's report?

Surprise, surprise! There's a literature that addresses this question - in the context of our own discipline, no less!

In particular, you might want to take a look at the various papers in the References section, below. I won't attempt to repeat what the authors have to say, except to suggest that there are a few things that you might consider.

If you're asked to referee a paper that's been submitted to an academic journals, then:

• I hope that you'll say "yes" (provided, of course, that you feel that you have the necessary expertise).
• If you have to decline, then I hope that you'll provide some suggestions for alternate referees.
• I hope that you'll put yourself on the other side of the fence, and be as constructive as you can be in your remarks and suggestions for the author(s) - even if you are recommending rejection. The careers of too many excellent young researchers have been stalled (or worse) by unduly negative and unhelpful feedback from condescending referees.
• I hope that you'll think carefully about the responsibility bestowed on you by the handling Editor/Associate Editor who's asked for your input. Your report should provide them with clear guidance as to how you feel about the submitted paper, and how you arrived at your recommendation regarding publishability.
• Don't include your recommendation to the Editor in the report that's to go to the author(s). This can put an Editor in a difficult spot, especially when the various referees differ in their recommendations,

Believe me, the person who asks for you time and advice as a referee is totally aware that they're imposing a huge imposition. You're busy, you don't need another request to give up your time and expertise, and sometimes it can seem like a pretty thankless task that you'll be taking on.

However, it's a two-way street. You know that. There's a lot of give-and-take, and goodwill, involved in the traditional peer-reviewing process that we use in our discipline. Please don't see it as a chore, but as a privilege. You're being asked to contribute to the advancement of knowledge, and provide a reciprocal service to you peers. On top of that, you get to read papers in your field that may not be readily available to others at that point in time.

References

Berk, J., C. R. Harvey, & D. A. Hirschleifer, 2016. Preparing a referee report: Guidelines and perspectives.

Berk, J. B., C. R. Harvey, & D. Hirschleifer, 2017. How to write an effective referee report and improve the scientific review process. Journal of Economic Perspectives, 31(1), 231-244.

Card, D. & S. DellaVigna, 2013. Nine facts about top journals in economics. NBER Working Paper No. 18665.

Hamermesh, D. S., 1992. The young economist’s guide to professional etiquette. Journal of Economic Perspectives, 6(1), 162-179.

Hamermesh, D. S., 1994. Facts and myths about refereeing. Journal of Economic Perspectives, 8(1), 153-163.

© 2018, David E. Giles