The Refereeing Process in Economics Journals

The peer-review process is an essential part of academic publishing. We use it in the hope of ensuring the honesty, novelty, importance, and timeliness of published research. The selection of (usually anonymous) referees by a representative of the journal to which a research paper has been submitted for consideration, and the preparation of the reports/reviews by those referees, are key steps in the overall process of the dissemination of research results.

There are several different "models" when it comes to the refereeing, or peer-review process. Some of these have been described and compared recently, and in detail, here. It's also interesting to note that peer-reviewing is actually a relatively recent phenomenon in most academic disciplines.

There's no doubt that a well-crafted referee's report is a blessing - to both the recipient author and the handling Editor/Associate Editor/Editorial Board member who's looking to that report for an informed basis for making an editorial decision.

Unfortunately, such reports are not necessarily the norm in Economics/Econometrics - more on this below!

I know this is so, all too well - not only from the times when, as an author, I've been "on the receiving end" of some decidedly unhelpful reports; but also (and much more importantly) from my experiences on the other side of the fence, as a "handling editor" for a quite a number of economics, econometrics, and statistics journals.

Some would say that the academic publishing process is a bit of a crap-shoot. At times, I think that there's some truth to that. However, there's a great deal that both authors and referees can do to make the exercise more rational. 

What's in a Journal Name?

Back in 2011 I put together a very light-hearted working paper titled, What's in a (Journal) Name? Here's the associated link.

That paper addressed the (obviously) important question: "Is there a a correlation between the ranking of an economics journal and the length of the journal's title?"

I analyzed a sample of 159 academic economics journals. Although there was no significant association between journal quality and journal title length for the full sample of data, I did find that there was a significant “bathtub” relationship between these variables when the data were subjected to a rank correlation analysis over sub-samples. 

This led me to conclude (p.5),among other things:

'This “bathtub” relationship will undoubtedly sound alarm bells in the corridors of publishing houses as they assess proposals for new economics journals. The title, Economics, is no longer available, having been cunningly snapped up in recent years by an open-access, open-assessment e-journal which managed to “cover all of the bases” in one fell swoop. Even more recently the American Economics Association laid claim to the titles Macroeconomics and Microeconomics, albeit with an “AEA” prefix that they may wish to re-consider. The publishers of the journal, SERIEs: Journal of the Spanish Economic Association, which was launched in 2010, will no doubt ponder the merits of dropping the last six words of its title. However, there is hope. The title Econometrica has been spoken for since 1933, but to the best of our knowledge the more worldly journal title Econometrics is still available. Publishers should register their interest forthwith!'

As usual the latter remark proved to be safe advice on my part! I wonder if my subsequent invitation to join the Editorial Board of Econometrics was some sort of reward? 

I'll probably never know!

© 2018, David E. Giles

The Series of Unsurprising Results in Economics (SURE)

Andrea Menclover of the University of Canterbury (New Zealand) has recently founded the SURE Journal, whose aims and scope are as follows:

'The Series of Unsurprising Results in Economics (SURE) is an e-journal of high-quality research with “unsurprising” findings. We publish scientifically important and carefully-executed studies with statistically insignificant or otherwise unsurprising results. Studies from all fields of Economics will be considered. SURE is an open-access journal and there are no submission charges. (My emphasis, DG.)

SURE benefits readers by:

• Mitigating the publication bias and thus complementing other journals in an effort to provide a complete account of the state of affairs;
• Serving as a repository of potential (and tentative) “dead ends” in Economics research.

SURE benefits writers by:

• Providing an outlet for interesting, high-quality, but “risky” (in terms of uncertain results) research projects;
• Decreasing incentives to data-mine, change theories and hypotheses ex post or exclusively focus on provocative topics.'

To find out more or to submit a manuscript, visit: http://surejournal.org.

This is a novel venture that has a lot to offer at a time when research replicability and publication bias are (rightly) receiving so much attention.

I'm delighted to be associated with the new journal as a member of its Editorial Board.

© 2018, David E. Giles

Suggested Reading for June

• Colussi, T., 2018. Social ties in academia: A friend is a treasure. Review of Economics and Statistics, 100, 45-50.
• Dette, H., K. Möllenhoff, S. Volgushev, & F. Bretz, 2018. Equivalence of regression curves. Journal of the American Statistical Association, online.
• Kumbhaker, S. C., C. F. Parmeter, & V. Zelenyuk, 2018. Stochastic frontier analysis: Foundations and advances. Working paper No. WP02/2018, Centre for Efficiency and productivity Analysis, School of Economics, University of Queensland.
• Morshed, A. A., E. Andreou, & O. Boldea, 2018. Structural break tests robust to regression misspecification. Econometrics, 6, 27.
• Zhang, L., 2018. Spurious regressions with high-order models: A reconsideration. Economics Letters, 168, 70-72.
• Zhao, S., A. Ullah, & X. Zhang, 2018. A class of model averaging estimators. Working Paper Series 18-11, Rimini Centre for Economic Analysis.

© 2018, David E. Giles

Econometrics Reading List for November

Some suggestions........

• Garcia, J. and D. E. Ramirez, 2017. The successive raising estimator and its relation with the ridge estimator. Communications in Statistics - Theory and Methods, 46, 11123-11142.
• Silva, I. R., 2017. On the correspondence between frequentist and Bayesian tests. Communications in Statistics - Theory and Methods, online.
• Steel, M. F. J., 2017. Model averaging and its use in economics. MPRA Paper No. 81568.
• Teräsvirta, T., 2017. Nonlinear models in macroeconometrics. CREATES Research Paper 2017-32.
• Witmer, J., 2017. Bayes and MCMC for undergraduates. American Statistician, 71, 259-274.
• Zimmerman, C., 2015. On the need for a replication journal. Federal Reserve Bank of St. Louis, Working Paper 2015-016A.

© 2017, David E. Giles

Marc Bellemare on "How to Publish in Academic Journals"

If you don't follow Marc Bellemare's blog, you should do.

And if you read only one other blog post this week, it should be this one from Marc, titled, "How to Publish in Academic Journals". Read his slides that are linked in the post.

Great advice that is totally applicable to anyone doing research in econometrics - theory or applied.

© 2017, David E. Giles

A "Journal of Insignificant (Economic) Results"?

The Replication Network carried a guest blog post by Andrea Menclova this week. The post was titled, "Is it Time for a Journal of Insignificant Results?"

I was previously unaware of the existence of such journals in Psychology, Biomedicine, and Ecology and Evolutionary Biology.

Andrea calls for the introduction of such a journal in Economics, and she makes a really good point.

Take a look at what she has to say!

© 2017, David E. Giles

Econometrics - Young Researcher Award

The journal, Econometrics, hasn't been around all that long, but it has published some great articles by some very prominent econometricians. And it's "open access" to readers, which is always good news.

Today, I received an email with the following important information:

"The journal Econometrics (http://www.mdpi.com/journal/econometrics) is inviting applications and nominations for the 2017 Young Researcher Award. The aim of the award is to encourage and motivate young
researchers in the field of econometrics.
Applications and nominations will be assessed by an evaluation committee chaired by the Editors and composed of Editorial Board Members.
Eligibility Criteria:

a) The upper age limit for the applicant is 40.
b) No more than 10 years since conferral of a PhD degree (by 30 June 2017).

The award will consist of: (1) a certificate; (2) an honorarium of 500 CHF; (3) a voucher for publishing two papers free of charge and without fixed deadlines in Econometrics if the Article Processing Charge will be applied; and (4) a £150 book voucher for PM book series sponsored by Palgrave Macmillan.

The application and nomination pack should include:

1. A Curriculum Vitae, including a complete list of publications and conference activities.

2. A description of the applicant’s major research contributions over the last 5 years, including clear discussions of 3 most representative publications published over the last 5 years. (For each publication, please provide significance of the publication and the applicant’s own contribution to the publication).

3. A letter of nomination from an established econometrician. The letter should highlight the candidate’s achievements and contribution to the field of econometrics.

Please send your application/nomination to the Econometrics Editorial Office at econometrics@mdpi.com by 30 June 2017. The winner will be announced on the Econometrics website in September 2017."

© 2017, David E. Giles

In Honour of Peter Schmidt

The latest issue of Econometric Reviews (Vol 36, Nos. 1-3) is devoted to papers that have been assembled to honour Peter Schmidt, of Michigan State University. Peter's contributions to econometrics have been outstanding, and it's great to see his work celebrated in this way.

In the abstract to their introduction to this collection Essie Maasoumi and Robin Sickles comment as follows:

"Peter Schmidt has been one of its best-known and most respected econometricians in the profession for four decades. He has brought his talents to many scholarly outlets and societies, and has played a foundational and constructive role in the development of the field of econometrics. Peter Schmidt has also served and led the development of Econometric Reviews since its inception in 1982. His judgment has always been fair, informed, clear, decisive, and constructive. Respect for ideas and scholarship of others, young and old, is second nature to him. This is the best of traits, and Peter serves as an uncommon example to us all. The seventeen articles that make up this Econometric Reviews Special Issue in Honor of Peter Schmidt represent the work of fifty of the very best econometricians in our profession. They honor Professor Schmidt’s lifelong accomplishments by providing fundamental research work that reflects many of the broad research themes that have distinguished his long and productive career. These include time series econometrics, panel data econometrics, and stochastic frontier production analysis."

I hope that you get a chance to read the papers in this issue of Econometric Reviews.

© 2017, David E. Giles

Reproducible Research in Statistics & Econometrics

The American Statistical Association has recently introduced reproducibility requirements for articles published in its flagship journal, The Journal of the American Statistical Association.

The following is extracted from p.17 of the July 2016 issue of Amstat News:

Coming from one of the most prestigious statistics journals, this is good news for everyone!

We could do with more of this in the econometrics journals, and in those economics journals that publish empirical studies. 

To that end, I again commend The Replication Network.

© 2017, David E. Giles

The Replication Network

This is a "shout out" for The Replication Network.

The full name is, The Replication Network: Furthering the Practice of Replication in Economics. I was alerted to TRN some time ago by co-organiser, Bob Reed, and I'm pleased to be a member.

What's TRN about:

"This website serves as a channel of communication to (i) update scholars about the state of replications in economics, and (ii) establish a network for the sharing  of information and ideas. 

The goal is to encourage economists and their journals to publish replications."

There's News & Events; Guest Blogs; Research involving replications in economics; and lots more.

Hats off to TRN. We need more of this!

© 2016, David E. Giles

A Pleasant Surprise

I could scarcely believe my good fortune when I opened the following email earlier today:

Dear Dr Giles,

Congratulations, your paper “Being ‘in’ assessment: The ontological layer(ing) of assessment practice” published in Journal of Applied Research in Higher Education has been selected by the journal’s editorial team as the Outstanding Paper of 2014. We aim to increase dissemination of such a high quality article as much as possible and aim to promote your paper by making it freely available for one year. I will confirm once the free access has gone live so that you will be able to let others know. This will be in the next couple of weeks. As a winner you will receive a certificate. Where possible, we like to organise for you to be presented with your certificate in person. We will be in touch with you shortly (next few weeks) in the hope that we can organize a presentation. Again, many congratulations on your award. We will be in touch with you regarding our plans to promote and present your award very soon. Please do not respond to this mail asking about when your chapter will be made freely available or about possible presentations. I will be in touch soon with all the details!! 

Best regards,
Jim Bowden

Academic Relations Manager | Emerald Group Publishing Limited 

Tel: +44 (0)1274 785013 | Fax: +44 (0)1274 785200

Now, I must confess that I had some difficulty recalling the details of what I'd written to deserve this unexpected honour. But it must have been pretty darned good!

So, to refresh my memory I took a quick look at the 2014 volume of the journal in question. (Bookmark this link for future free access!)

Sure enough, there I it was - not the lead article, but close enough (apparently):

For reasons that I simply can't fathom, our library doesn't subscribe to this journal. For reasons that I definitely can fathom, neither do I! The promised free access has not yet "gone live", so I'll have to spare you the pleasure of a replication of the full text of the article here.

However, by way of recompense, here's the abstract:

Abstract:

Purpose

– Current discourses on educational assessment focus on the priority of learning. While this intent is invariably played out in classroom practice, a consideration of the ontological nature of assessment practice opens understandings which show the experiential nature of “being in assessment”. The purpose of this paper is to discuss these issues.

Design/methodology/approach

– Using interpretive and hermeneutic analyses within a phenomenological inquiry, experiential accounts of the nature of assessment are worked for their emergent and ontological themes.

Findings

– These stories show the ontological nature of assessment as a matter of being in assessment in an embodied and holistic way.

Originality/value

– Importantly, the nature of a teacher's way-of-being matters to assessment practices. Implications exist for teacher educators and teacher education programmes in relation to the priority of experiential stories for understanding assessment practice, the need for re-balancing a concern for professional knowledge and practice with a students’ way of being in assessment, and the pedagogical implications of evoking sensitivities in assessment.

I can hardly believe I wrote that!

(I have replied to Mr. Bowden, expressing my gratitude but suggesting that a certain David Giles in the School of Education at Flinders University in Australia may be even more pleased to hear from him than I was.)

© 2015, David E. Giles

When Did You Last Check Your Code?

Chris Blattman (Columbua U.) has a blog directed towards international development, economics, politics, and policy.

In a post yesterday, Chris asks: "What happens when a very good political science journal checks the statistical code of its submissions?"

The answer is not pretty!

His post relates to the practice of the Quarterly Journal of Political Science to subject empirical papers to in-house replication. This involves running the code provided by authors. He cites a batch of 24 such papers in which only 4 were found to be error-free.

Have you checked you own code recently?

© 2014, David E. Giles

Marc Bellemare on Social Media

Marc Bellemare has been catching my attention recently. On Saturday I had a post that mentioned his talk on "How to Publish Academic Papers". I know that a lot of you have followed this up already.

Today, I just have to mention another of his talks, given last Friday, titled "Social Media for (Academic) Economists". Check out his blog post about the talk, and then look at this slides that are linked there.

Yep, I agree with pretty much everything he has to say. And nope, we're not related!

© 2014, David E. Giles

Advice on Publishing

I've put in a lot of time over the years as an Editor, Associate Editor, or Editorial Board member, for a number of economics and statistics journals, ranging from Journal of Econometrics and Econometric Theory, to Journal of International Trade & Economic Development.  I've also refereed more papers than care to think about. 

Students, rightly, are eager to get the scoop on how to get their work published in good journals. They often talk to me about this. My suggestion would be to read, and follow the advice given by Marc Bellemare in his talk, "How to Publish Academic Papers". 

Just do it!

(HT to David Stern for unwittingly making me aware of Marc's talk,)



© 2014, David E. Giles

Thanks for Downloading!

In an earlier post I mentioned a paper that I co-authored with Xiao Ling. The paper is "Bias reduction for the maximum likelihood estimator of the parameters of the generalized Rayleigh family of distributions. Communications in Statistics - Theory and Methods, 2014, 43, 1778-1792.

Over the period January to July 2014, this paper was downloaded 144 times from the journal's website. That made it the 6th most downloaded paper for that period - out of all papers downloaded from all volumes/issues of Communications in Statistics - Theory and Methods.

My guess is that some of you were responsible for this. Thanks!

© 2014, David E. Giles

Promoting Econometrics Through Econometrica

Regular readers of this blog will know that I have an interest (but negligible talent or authority) in the history of econometrics. Actually, the same applies to the history of the discipline of statistics. I find it difficult to appreciate where we are without knowing something about where we came from, and I try to convey this to my students.

Olav Bjerkholt (University of Oslo) has provided me with a lot of very valuable material and insights in recent months, and I've been delighted to have drawn on his contributions in previous posts (e.g., here, here, and here).

Olav wrote to me yesterday, as follows:

Rejected Economists

If you're feeling "down" as a result of a recent rejection of your work by a nasty journal editor, you may find some comfort in this paper from The Journal of Economic Perspectives in 1994: "How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists", by Joshua Gans and George Shepherd.

In it, you can read about the many famous economists - Nobel laureates included - who have struggled to get their work published. There are some great stories here.

In the process, you'll learn why the Tobit model is so-named - and it's not just for the obvious reason that comes to mind!

I promise that you'll feel much better after reading what Gans and Shepherd have to say.

.

© 2014, David E. Giles

An Extreme Publication Lag

We all complain about the delays associated with the academic publishing process. The referees can be very slow in reaching their recommendations; the revisions sometimes seem to be interminable; and then the accepted paper sits in a long queue awaiting its grand entry onto the world stage.

Occasionally - very occasionally - we encounter an extreme outlier in this process. I seem to recall that there was a paper by Paul Samuelson, written in the 1940's, that eventually appeared in print some decades later.

Olav Bjerkholt has kindly drawn my attention to an exceptional econometric example of delayed publication, involving an important paper by one of our founding fathers - Trygve Haavelmo. (Olav has written extensively and authoritatively on the early history of econometrics, and I've mentioned some of his contributions previously - here, and here.)

Now, what about this publishing delay?

Replication in Economics

I was pleased to receive an email today, alerting me to the "Replication in Economics" wiki at the University of Göttingen:

"My name is Jan H. Höffler, I have been working on a replication project funded by the Institute for New Economic Thinking during the last two years and found your blog that I find very interesting. I like very much that you link to data and code related to what you write about. I thought you might be interested in the following:   We developed a wiki website that serves as a database of empirical studies, the availability of replication material for them and of replication studies: http://replication.uni-goettingen.de
It can help for research as well as for teaching replication to students. We taught seminars at several faculties internationally - also in Canada, at UofT - for which the information of this database was used. In the starting phase the focus was on some leading journals in economics, and we now cover more than 1800 empirical studies and 142 replications. Replication results can be published as replication working papers of the University of Göttingen's Center for Statistics.
Teaching and providing access to information will raise awareness for the need for replications, provide a basis for research about the reasons why replications so often fail and how this can be changed, and educate future generations of economists about how to make research replicable.
I would be very grateful if you could take a look at our website, give us feedback, register and vote which studies should be replicated – votes are anonymous. If you could also help us to spread the message about this project, this would be most appreciated."

I'm more than happy to spread the word, Jan. I've requested an account, and I'll definitely be getting involved with your project. This look like a great venture!

© 2014, David E. Giles