Economic Argument for Austerity Based on Excel Error?

[Jacob]

People better versed in economics can perhaps confirm or debunk this, but apparently the primary academic and data-based argument for austerity and against a high debt-to-GDP ratio was based on an excel error. There are some arguments against the methodology as well.

Link to argument here: http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems#.UW147o4A23o.twitter

[Jacob]

Summary:

In 2010, economists Carmen Reinhart and Kenneth Rogoff released a paper, "Growth in a Time of Debt." Their "main result is that...median growth rates for countries with public debt over 90 percent of GDP are roughly one percent lower than otherwise; average (mean) growth rates are several percent lower." Countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate, in fact.
...
They find that three main issues stand out. First, Reinhart and Rogoff selectively exclude years of high debt and average growth. Second, they use a debatable method to weight the countries. Third, there also appears to be a coding error that excludes high-debt and average-growth countries. All three bias in favor of their result, and without them you don't get their controversial result.

[MadImmortalMan]

Been following this since this morning. Basically, it's a nipick that alters the numbers slightly due to four or five countries' data from the immediate post-ww2 years not being included in the sample data. Canada, Australia, New Zealand..I forget the others. It's pretty insignificant.

Edit: http://www.businessinsider.com/carmen-reinhart-table-counters-critique-2013-4

[MadImmortalMan]

Here's the original critique:

Selective Exclusions. Reinhart-Rogoff use 1946-2009 as their period, with the main difference among countries being their starting year. In their data set, there are 110 years of data available for countries that have a debt/GDP over 90 percent, but they only use 96 of those years. The paper didn't disclose which years they excluded or why.

Herndon-Ash-Pollin find that they exclude Australia (1946-1950), New Zealand (1946-1949), and Canada (1946-1950). This has consequences, as these countries have high-debt and solid growth. Canada had debt-to-GDP over 90 percent during this period and 3 percent growth. New Zealand had a debt/GDP over 90 percent from 1946-1951. If you use the average growth rate across all those years it is 2.58 percent. If you only use the last year, as Reinhart-Rogoff does, it has a growth rate of -7.6 percent. That's a big difference, especially considering how they weigh the countries.

Dunno if you'd count those years as an anomaly or not. I guess it was just the three countries.

[Sheilbh]

From Jacob's source, this doesn't seem really insignificant in the context of their paper:

Selective Exclusions. Reinhart-Rogoff use 1946-2009 as their period, with the main difference among countries being their starting year. In their data set, there are 110 years of data available for countries that have a debt/GDP over 90 percent, but they only use 96 of those years. The paper didn't disclose which years they excluded or why.

Herndon-Ash-Pollin find that they exclude Australia (1946-1950), New Zealand (1946-1949), and Canada (1946-1950). This has consequences, as these countries have high-debt and solid growth. Canada had debt-to-GDP over 90 percent during this period and 3 percent growth. New Zealand had a debt/GDP over 90 percent from 1946-1951. If you use the average growth rate across all those years it is 2.58 percent. If you only use the last year, as Reinhart-Rogoff does, it has a growth rate of -7.6 percent. That's a big difference, especially considering how they weigh the countries.

Unconventional Weighting. Reinhart-Rogoff divides country years into debt-to-GDP buckets. They then take the average real growth for each country within the buckets. So the growth rate of the 19 years that England is above 90 percent debt-to-GDP are averaged into one number. These country numbers are then averaged, equally by country, to calculate the average real GDP growth weight.

In case that didn't make sense let's look at an example. England has 19 years (1946-1964) above 90 percent debt-to-GDP with an average 2.4 percent growth rate. New Zealand has one year in their sample above 90 percent debt-to-GDP with a growth rate of -7.6. These two numbers, 2.4 and -7.6 percent, are given equal weight in the final calculation, as they average the countries equally. Even though there are 19 times as many data points for England.

Now maybe you don't want to give equal weighting to years (technical aside: Herndon-Ash-Pollin bring up serial correlation as a possibility). Perhaps you want to take episodes. But this weighting significantly reduces the average; if you weight by the number of years you find a higher growth rate above 90 percent. Reinhart-Rogoff don't discuss this methodology, either the fact that they are weighing this way or the justification for it, in their paper.

Coding Error. As Herndon-Ash-Pollin puts it: "A coding error in the RR working spreadsheet entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis. [Reinhart-Rogoff] averaged cells in lines 30 to 44 instead of lines 30 to 49...This spreadsheet error...is responsible for a -0.3 percentage-point error in RR's published average real GDP growth in the highest public debt/GDP category." Belgium, in particular, has 26 years with debt-to-GDP above 90 percent, with an average growth rate of 2.6 percent (though this is only counted as one total point due to the weighting above).

Being a bit of a doubting Thomas on this coding error, I wouldn't believe unless I touched the digital Excel wound myself. One of the authors was able to show me that, and here it is. You can see the Excel blue-box for formulas missing some data:

This error is needed to get the results they published, and it would go a long way to explaining why it has been impossible for others to replicate these results. If this error turns out to be an actual mistake Reinhart-Rogoff made, well, all I can hope is that future historians note that one of the core empirical points providing the intellectual foundation for the global move to austerity in the early 2010s was based on someone accidentally not updating a row formula in Excel.

I'm not clear how the coding errors slipped by, I thought this sort of data was available when reviewing academic articles?

Their response:

We literally  just received this draft comment, and will review it in due course.   On a cursory look, it seems that that Herndon Ash and Pollen also find lower growth when debt is over 90% (they find 0-30 debt/GDP , 4.2% growth;   30-60, 3.1 %;  60-90, 3.2%,;  90-120, 2.4% and over 120, 1.6%).  These results are, in fact, of a similar order of magnitude to the detailed country by country results we present in table 1 of the AER paper, and to the median results in Figure 2.  And they are similar to estimates in much of the large and growing  literature,  including our own attached August 2012 Journal of Economic Perspectives paper (joint with Vincent Reinhart) .  However, these strong similarities are not what these authors choose to emphasize.

The  2012 JEP paper largely anticipates and addresses any concerns about aggregation (the main bone of contention here), The JEP paper  not only provides individual country averages (as we already featured in Table 1 of the 2010 AER paper)  but it goes further and provide episode by episode averages.  Not surprisingly, the results are broadly similar to our original 2010 AER table 1 averages and to the median results that also figure prominently..  It is hard to see how one can interpret these tables and individual country results as showing that public debt overhang over 90% is clearly benign.

The JEP paper with Vincent Reinhart looks at all public debt overhang episodes for advanced countries in our database, dating back to 1800.  The overall average result shows that public debt overhang episodes (over 90% GDP for five years or more) are associated with 1.2%  lower growth as compared to  growth when debt is under 90%.  (We also include in our tables the small number of shorter episodes.)  Note that because the historical public debt overhang episodes last an average of over 20 years, the cumulative effects of small growth differences are potentially quite large.  It is utterly misleading to speak of a 1% growth differential that lasts 10-25 years as small.

By the way, we are very careful in all our papers to speak of "association" and not "causality" since of course our 2009 book  THIS TIME IS DIFFERENT showed that debt explodes in the immediate aftermath of financial crises.  This is why we restrict attention to longer debt overhang periods in the JEP paper., though as noted there are only a very limited number of short ones.   Moreover, we have generally emphasized the 1% differential median result in all our discussions and subsequent writing, precisely to be understated and cautious , and also in recognition of the results in our core Table 1 (AER paper).

Lastly, our 2012 JEP paper cites papers from the BIS, IMF and OECD (among others) which virtually all find very similar conclusions to original findings, albeit with slight differences in threshold, and many nuances  of alternative interpretation..  These later papers, by they way,  use a variety of methodologies for dealing with non-linearity and also for trying to determine causation.  Of course much further research is needed as the data we developed and is being used in these studies is new.  Nevertheless, the weight of the evidence to date –including this latest comment -- seems entirely consistent with our original interpretation of the data in our 2010 AER paper.

[Admiral Yi]

The critique completely misses the point of austerity.

You don't reduce your deficit because doing so improves your growth (although the crowding out effect should have that effect); you reduce your deficit because if you don't people will start charging you more interest and eventually stop lending to you altogether.

[Jacob]

The critique completely misses the point of austerity.

You don't reduce your deficit because doing so improves your growth (although the crowding out effect should have that effect); you reduce your deficit because if you don't people will start charging you more interest and eventually stop lending to you altogether.

If that's the argument, why are people arguing for austerity given the record low levels of interest right now? Should we wait until rates start moving up a bit first?

[viper37]

If that's the argument, why are people arguing for austerity given the record low levels of interest right now? Should we wait until rates start moving up a bit first?

There are a lot of good reasons to wait later for austerity.  In fact, it's counter-productive for the economy to apply austerity measures right now as it is slowing growth.

However, we've been there before. 

We had a period of low interest rates and economic growth, we (occidental countries) spent even more money instead of paying back our debt.

We had a perdiod of high interest rates following a recession, we spent even more money on interest rates and all kinds of social programs instead of paying back the debt.

So if we don't do it now, when we are forced to do so, when are we going to do it?  When things start to get better and nobody really worries about our debt to gdp ratio?  Of course not, it won't be a problem then.  We'll wait for the next crisis, when lenders don't want to lend money, and things are gonna be even worst than now.

[Sheilbh]

I think that this research is overstated. Austerity or not's an ideological and political argument and I think studies like this have been really more rhetorical weapons than anything.

If this paper's been key to someone's economic thinking and argument over the last few years (Osborne springs to mind) then this is a problem. But I don't think that covers many people.

But even that's an example of what I mean. When Osborne started he cited this paper a lot and the IMF's advice (and praise) for his fiscal consolidation plan. Now this paper's not so solid and the IMF have recently named Britain as a country with 'fiscal space' that should slow the pace of consolidation. That won't make him change his policy because it's as much about politics as economics and he can't be seen to u-turn, especially when prioritising rapid deficit reduction was the key idea that held the coalition together.

You don't reduce your deficit because doing so improves your growth (although the crowding out effect should have that effect); you reduce your deficit because if you don't people will start charging you more interest and eventually stop lending to you altogether.

Nonsense. That explains - to some extent - some peripheral nations in Europe. It doesn't explain the US, the UK, the Netherlands, France or even Germany. I'd be willing to bet the majority of austerity over the last 3 years has been in countries with historically low interest rates.

[MadImmortalMan]

Just to be clear, the original paper does not advocate austerity.

[CountDeMoney]

I think that this research is overstated. Austerity or not's an ideological and political argument and I think studies like this have been really more rhetorical weapons than anything.

It's a constructed argument that aligns very well with the destruction-of-big-gubbmint types, that's all it is. 

[Razgovory]

The critique completely misses the point of austerity.

You don't reduce your deficit because doing so improves your growth (although the crowding out effect should have that effect); you reduce your deficit because if you don't people will start charging you more interest and eventually stop lending to you altogether.

If that's the argument, why are people arguing for austerity given the record low levels of interest right now? Should we wait until rates start moving up a bit first?

Cause it didn't work.

[Admiral Yi]

If that's the argument, why are people arguing for austerity given the record low levels of interest right now? Should we wait until rates start moving up a bit first?

People are arguing for austerity now because of very high debt/GDP ratios.  And because of the moral argument of spending money that someone else will have to pay off in the future.

Rates don't move up a bit.  Investor perception of credit worthiness is flat until it reaches a tipping point, when things go all to shit.

[Admiral Yi]

Nonsense. That explains - to some extent - some peripheral nations in Europe. It doesn't explain the US, the UK, the Netherlands, France or even Germany. I'd be willing to bet the majority of austerity over the last 3 years has been in countries with historically low interest rates.

What do historically low rates have to do with anything at all?  Do you think those countries you mentioned were granted exemptions from sovereign risk analysis?

[Sheilbh]

What do historically low rates have to do with anything at all?  Do you think those countries you mentioned were granted exemptions from sovereign risk analysis?

I don't think rates are generally reflecting risk.