Lukas Püttmann    About    Research    Blog

Cherry blossoms in Kyoto, 801-2015

Cosma Shalizi offers a course with great name “Data over Space and Time”. As part of this course he uses data on the first day in the year that the cherry trees in Kyoto started blossoming. This data was collected by Yasuyuki Aono and coauthors.

Let’s check it out:

library(tidyverse)

kyoto <- read_csv("http://www.stat.cmu.edu/~cshalizi/dst/18/data/kyoto.csv")

ggplot(kyoto, aes(Year.AD, Flowering.DOY)) +
  geom_line() +
  labs(title = "Cherry Blossoms in Kyoto",
       subtitle = "801-2015",
       x = "Year",
       y = "Day in year of full flowering") +
  geom_smooth()
Cherry Blossoms in Kyoto by day in year since year 801 until year 2015

Note the earlier blossoming of trees at the end of the sample which is a sign of rising temperatures.

Collected links

  1. Against home cooking, by Rachel Laudan
  2. Every graph in the AER since 1911 and paper. I’ve also found that economists heavy use of linecharts is quite atypical. Outside academia, other formats such as barcharts are much more common.
  3. I was very impressed by this podcast episode of 80,000 hours interviewing Eva Vivalt. I didn’t expect to find meta-analyses interesting, but they are! One of my favorite bits of insight: If we asked researchers about their priors about what the results of a research project will be (DellaVigna-Pope style), then null results suddenly become interesting and worthy to publish. I also liked this blog post by Ricardo Dahis discussing the framework for meta analyses.
  4. Simon Kucinskas: “How to Measure Biased Reaction to News
  5. A short radio review (in German) of Konrad Biermann’s Konrad Adeanuer biography. I liked the book, but found it speculative and thin on facts.
  6. 1979
  7. Movie reviews
  8. Loriot (in German)

Enlighten Me How?

I.

Sometimes a book is so good so that I really want to write a review, but I’m not sure I have anything substantial to add. That was the case for me reading Steven Pinker’s new book “Enlightenment Now: The Case for Reason, Science, Humanism and Progress”. There’s very little in his chain of argumentation that I could attack. Very frequently, I thought, “But how about X?” and that X was usually what Pinker addressed next.

In the book, Pinker argues that humanity has made progress in almost any measurable way. In his previous book, “The Better Angels of Our Nature”, he showed that the prevalence of violence in human lives has fallen strongly over the course of history and that this trend continues.1 In the new book, he widens his argument to include all areas of what he often calls “human flourishing”: Poverty has fallen and people live longer, healthier and happier lives. Wars have become more rare, existential risks are manageable, the world is becoming more democratic and we can handle risks to our ecological system. The coverage of civil rights has kept expanding to include women and minorities.

The book is dense in data and well argued. I really enjoyed it and agree with most of what the author writes. Most of the trends I was aware of, but I found his discussions of some issues (inequality, ecology, nuclear energy) refreshingly different from the takes I usually read (in particular in Germany) and well-founded. I hope this disclaimer removes the negativity bias (also discussed in the book) from the rest of this review.

II.

In a book that covers almost everything, it’s easy to attack something that’s missing.2 I want to focus my review on the core of Pinker’s argument which is (all the keywords are in the subtitle to his book):

  1. Things have gotten better (progress!) for humanity in almost all measurable ways.
  2. That’s because humans use reason and science to accumulate more knowledge and solve problems.
  3. Associated with the increase of rational thought in our lives is a rise in atheist internationalist altruism (humanism).
  4. Because this system of belief of reason/science with humanism has yielded the best results for humanity, we we should celebrate it and promote it further.

The twist in his argument is that there’s a divine coincidence: The institutions and the belief system that we like best (rational thought, a reduction of the role of religion in our lives) just happen to be what achieves the best outcomes.

But that’s a shallow defense of democracy and civil liberties.3 Aren’t there values and principles that we hold dear, even if they did not lead to the best economic outcomes?

Henry Kissinger defended the West’s personal freedom, democracy and free markets during the Cold War by arguing that this is the better system of belief, even if it hadn’t lead to better economic outcomes than communism (Ferguson, 2016). At the time, the alternative that communist countries might grow more than free-market economies was taken seriously. Even leading economists expected the communist countries to overtake the Western countries in economic terms eventually.

A striking twist is then the rise of China. This is a country without democracy that has pulled off an incredible reduction of human poverty for a huge number of people. Using Pinker’s line of argument that should lead us to take authoritarianism seriously as a morally correct system.4

The problem is that China introduced economic liberties, but not political liberties. On the other side of the spectrum is Soviet Russia under Gorbatshev. William Taubmann’s “Gorbachev: His Life and Times” chronicles very well Gorbachev worked hard to introduce political freedom in the Soviet Union, but didn’t recognize the importance of economic freedom. The communist leadership was well aware of the shortage of consumption goods and the massive inefficiencies in the Soviet economy, but Gorbachev considered political freedom more important and didn’t trust free markets. It’s not that Gorbachev was ignorant of the economic troubles, but that the solution didn’t seem to be a liberalization of economic life.

Sadly, neither China or Russia delivered real freedom. So what do we conclude? People in China and Russia are better off than they were some decades ago and that’s something to applaude.

But I think the West’s model of democracy and personal freedom are superior, if they didn’t yield the best outcomes. Milton Friedman discusses this in depth in “Capitalism and Freedom”. He argues that economic and political freedom go together, that only if you are economically free you can hold views that are different to those of the government. He gives the example of a dissident who – in a centralized system – couldn’t even finance his daily life or his campaigning. This would suggest that economic freedom might beget political freedom. But as the case of China shows, that’s not always the case.

In the end, Friedman does not argue from economic or material grounds, but in terms of ideology:

The heart of the liberal philosophy is a belief in the dignity of the individual, […]. This implies a belief in the equality of men in one sense; in their inequality in another. […] The liberal will therefore distinguish sharply between equality of rights and equality of opportunity, on the one hand, and material equality or equality of outcome on the other. He may welcome the fact that a free society in fact tends toward greater material equality than any other yet tried. But he will regard this as a desirable by-product of a free society, not its major justification. (“Capitalism and Freedom”, Chapter 12: “Alleviation of Poverty”)

And Pinker falls into exactly that trap in this book.

III.

Pinker likes to cite internationalism as a goal, that obviously it’s irrational to weight people’s well-being less than that of people in your country. But that’s not obvious to most people and that’s not how most people act.

Pinker writes (my emphasis):

Ideologies that justify violence against innocents, such as militant religions, nationalism, and Marxism, can be countered with better systems of value and belief. (Chapter 23)

But it’s just the problem of belief systems that they determine what’s good and bad. It’s a circular argument to use evidence that there’s lot of progress according to and due to that value system to make a case for exactly that value system.

Say, your value system tells you that you want to maximize the number of paper clips in existence on earth. So if the number of paperclips in existence has increased: Voilá - that belief system is great!

But what is the right value system – “that is exactly the question”.

IV.

A response to my criticism might be the following: Reason and science are critically not a belief system, but the correct way of learning about the world.

OK, granted. But humanism is a belief and value system. And scoring humanity’s achievements yields very different assessments through the lens of humanism than it would through alternative world views. If your religion tells you that life starts with the inception, then the increase in pregnancy terminations is an incredible disaster. If you’re a nationalist who believes that people’s and races are a thing and should be kept separate, then globalization, intermarriage and the rise in cosmopolitanism are a big problem.

Pinker singles out those two world views (religion and romantic-nationalism) and takes them apart one by one. He writes that there is “probably no god”, so basing our morality on religion leads us to the wrong place. He writes that romantic-nationalist inspired by Nietzsche misunderstand human nature.

And maybe he finally wins that way, because we can reason about our value systems and test the premises they are built on.

V.

This book is a clear-sighted summary of the achievements of economic growth. It synthesizes vast amounts of data and research and makes a much more grand case than comparable books.5

Sometimes a book is so good so that I really want to write a review, but I’m not sure I have anything substantial to add. Arguing for humanism on materialistic grounds bothers me, but it might be our best argument for it. Let’s be happy about the divine coincidence.

References

Carroll, Sean (2016). The Big Picture: On the Origins of Life, Meaning, and the Universe Itself. Dutton.

Ferguson, Niall (2016). Kissinger: 1923-1968: The Idealist. Penguin.

Friedman, Milton (1962). Capitalism and Freedom. University of Chicago Press.

Pinker, Steven (2012). The Better Angels of Our Nature: Why Violence Has Declined. Penguin.

Pinker, Steven (2018). Enlightenment Now: The Case for Reason, Science, Humanism, and Progress. Viking.

Taubmann, William (2017). Gorbachev: His Life and Times. Simon & Schuster.


  1. And he showed what great outliers the two World Words were. 

  2. For example, why isn’t the extinction of species covered? Isn’t that a case where there’s clearly no progress, but instead our economic expansion has done harm? 

  3. Pinker also makes another case for democracy which is that it smooths out errors by individuals well. But that argument feels tacked on to his main point and is not supported by the same level of data and explanation as the main thrust of his argument. 

  4. Pinker being Pinker he obviously also covers that case and writes that China today is already quite a more liberal and open society compared to some decades ago. But China is not a free, democratic country (Freedom House, for example, classifies it as “not Free”). 

  5. In particular, I find it a great complement to Sean Carroll’s “The Big Picture”. 

Alternatives

There is a story about an old man which rheumatism who asked his healthy friend how he managed to avoid the malady.
“By taking a cold shower every morning all my life” was the answer.
“Oh! exclaimed the first, “then you had cold showers instead.”

That’s from “One Two Three … Infinity”, by George Gamow, first published in 1947.

Inventing code

This recent paper in Nature Communications is a really cool (source):

We analysed a database of 21,745,538 lines of computer code in total and 483,173 unique lines, originating from 47,967 entries to 19 online collaborative programming competitions organised over the course of 14 years by the MathWorks software company. In every contest, the organisers set a computational challenge and, over the course of one week, participants developed and provided solutions in the form of MATLAB® code.

Figure 3 is amazing.

Collected links

  1. Interview with the number one ranked Kaggler (recommended)
  2. Jon Schwabish redesigns a table
  3. Fantastic explainer for copulas
  4. Interesting figure
  5. The Economist writes:

    Shipping a 70kg parcel from Shanghai to London with DHL Express takes three times longer, and costs four times as much, as buying a human of the same weight an airline ticket. The passenger gets a baggage allowance and free drinks, too.

  6. Bundesrechnungshof (German Federal Audit Office) laments (in German) the fact that Germany subsidizes the tabaco industry. On average, the 11,000 employees receive 600 cigarettes per month. This was introduced after World War One to reduce stealing by employees.
  7. German Empire and the 2007 Polish elections

"Better Presentations", by Jonathan Schwabish

Jonathan Schwabish’s book “Better Presentations” is very good. The author wrote this paper in the Journal of Economic Perspectives some years ago and I really enjoyed that then. The book expands on the themes of the paper.

He emphasizes the need to better tailor presentations to the audience. He has a simple set of notes to get one started (see the “Better Presentations Worksheet” here). A presentation in a research seminar can go into great depth, but a presentation at a large diverse conference or a presentation to funding providers should not. The less well acquainted people are with the topic, the more important is providing context about one’s research.

The most important point Schwabish makes is his explanation of the pyramid, the inverted pyramid and the hourglass. The pyramid is the traditional form of presentation where you first set the stage by mentioning your research hypothesis, review the literature, then introduce the data, explain your methods and - last - reveal what you find.

Most researchers have by now realized that this is not a good idea. People don’t know where the presentation is headed and what’s important. An alternative model comes from journalism: The inverted pyramid. Start with a bang and tell the reader all about what you’ve found out. And then you keep adding more details later on.

But, as the author points out, you can do both and combine the two to an hourglass structure. You start broad, provide context for who you are, what you’re working on and why. You set the right expectations of what your paper is about. Then go into more details and explain exactly what you’re doing. In the end, you zoom out again: What did we just learn that’s new and why does it matter?

I watch too many people run out of time at the end of their presentation and say, “Let me just quickly show you these two regression tables of robustness tests.” No! The end of your presentation is too important to waste it on a regression table. The audience is most attentive at the beginning and end, so don’t waste those moments on details.

I also learned a number of smaller points:

  • Get rid of bullet points and just start items directly.
  • Cooler colors (blue, green) pull away from you into the page, warmer colors (red, yellow) pop out.
  • Schwabish refutes the idea that you should only have X slides per Y minutes. He considers such metrics too crude and thinks it leads to bad design. It encourages people to put too much text on slides. It’s better to split thoughts into more slides because that’s what slide shows excel at.
  • He recommends breaking the white we normally use as slide background to a light gray. I’ve tried that, but it leads to other problems: Our statistical software draws graphs with white backgrounds, so figures stand out against the gray background and it looks strange. I think it’s overkill to change the background color in every figure we draw and it’s better to just continue using white as the background color.1

In economics, most of us use Latex Beamer and I don’t think we’re doing us a favor with that. It’s great to write equations, so maybe it’s the right choice for theory papers.2 But the majority of papers nowadays have a focus on empirics and include many figures and for these it’s not a good tool: Just try moving your Latex figure a little to the right, try making it full size or try adding a little text on top of your figure. And everything takes so much longer! You type \textit{} for italics and then have to watch your Latex compile … and fail because you’ve mistyped something or have forgotten a curly bracket.

I switched back to PowerPoint a year ago and I much prefer that now. The most tricky thing I find including regression tables from Latex.3 But most of those probably shouldn’t be in our presentations anyway and those that we do include can do with much less information (no standard errors, covariates, R², …) than we put into our papers. In conclude that using Latex Beamer is mostly signaling.

Anyway, the book by Schwabish is great and I recommend it.

References

Schwabish, Jonathan A. (2014). “An Economist’s Guide to Visualizing Data”. Journal of Economic Perspectives, 28(1): 209-234.

Schwabish, Jonathan A. (2016). Better Presentations: A Guide for Scholars, Researchers, and Wonks. Columbia University Press. (link)

  1. Eric points out that (in R) you can use no background color, by adding + theme(plot.background = element_blank(), panel.background = element_blank()) to your ggplot(...)

  2. Whether slide shows are the appropriate way to explain complicated theory is another question. For lectures, I’m positively convinced that it’s inferior to writing equations on the board, as that leads to a slower and more phased-in introduction of new material. 

  3. I now use the for-pay version of Adobe to convert regression tables from pdf to word. In word I can change fonts, then I copy tables and paste them into PowerPoint with right click “Picture”. This works perfectly for me. 

H(el)P(ful) filtering

I.

Separating the cycle from the trend of a time series is a common activity for macroeconomists. The choice of how to detrend variables can have major effects on firms and the economy as a whole. For example, the “Credit-to-GDP gaps” by the BIS are used by financial market regulators to decide on how to adjust counter-cyclical capital buffers which could mean that banks have to issue more capital. Estimating a different cycle can therefore directly affect bank profitability.

The usual method to detrend macroeconomic variables is the Hodrick-Prescott (HP) filter. The HP filter is nice, because it’s easy to understand and implement. The choice of allows using outside knowledge to set a cycle length that we consider reasonable.

But along came James Hamilton who wrote a paper last year called “Why you should never use the Hodrick-Prescott filter” (published, voxeu, pdf). He criticizes that the HP filter has no theoretical foundation and that it doesn’t accomplish what we would like it to do.

Hamilton proposes a better way to detrend. First, you choose an appropriate cycle length. Then - for every point in time - the trend is the value you would have predicted one cycle ago with a linear regression. To appropriately model time series behavior and (as a nice side effect) capture seasonalities, you pick multiples of 4 for quarterly and of 12 for monthly frequencies.

In this post, I’ll show how to implement the new Hamilton detrending method and compare it with the HP filter. I’ll take one of the variables that Hamilton uses in his paper and get exactly the same values.

II.

First, load some packages (see here how to use FredR):

library(tidyverse)
library(data.table)
library(zoo)
library(lubridate)
library(ggthemes)
library(FredR)
library(modelr)
library(mFilter)

Insert your FRED API key below:

api_key <- "yourkeyhere"
fred    <- FredR(api_key)

Get nonfarm employment data from Fred. We keep only end of quarter values to use the same sample as Hamilton (first quarter of 1947 to last quarter of 2016). And we transform the data the same way (“100 times the log of end-of-quarter values”):

df <- fred$series.observations(series_id = "PAYEMS") %>%
  mutate(date = as.yearmon(date)) %>% 
  select(date, emp = value) %>% 
  mutate(emp = 100 * log(as.numeric(emp))) %>% 
  filter(month(date, label = TRUE) %in% c("Mar", "Jun", "Sep", "Dec")) %>% 
  mutate(date = as.yearqtr(date)) %>% 
  filter(date >= as.yearqtr("1947 Q1"),
         date <= as.yearqtr("2016 Q2"))

We use the HP filter from the mFilter package like this:

hp <- hpfilter(df$emp, freq = 1600, type = "lambda")

df <- df %>% 
  mutate(hp_trnd = c(hp$trend),
         hp_cl = hp$cycle)

The dataframe then looks like this:

df

# A tibble: 278 x 4
# date            emp hp_trnd  hp_cl
# <S3: yearqtr> <dbl>   <dbl>  <dbl>
# 1 1947 Q1       1068.   1068.  0.407
# 2 1947 Q2       1069.   1068.  0.458
# 3 1947 Q3       1070.   1069.  0.939
# 4 1947 Q4       1071.   1069.  1.38 
# 5 1948 Q1       1071.   1070.  1.19 
# 6 1948 Q2       1072.   1070.  1.56 
# 7 1948 Q3       1072.   1070.  1.71 
# 8 1948 Q4       1072.   1071.  0.688
# 9 1949 Q1       1070.   1071. -1.53 
# 10 1949 Q2       1069.   1072. -3.14 
# ... with 268 more rows
     

Get NBER recession dummies:

usrec <- fred$series.observations(series_id = "USREC") %>%
  mutate(date = as.yearmon(date)) %>% 
  select(date, rec = value) %>%   
  mutate(date = as.yearqtr(date)) %>% 
  filter(date >= as.yearqtr("1947 Q1"),
         date <= as.yearqtr("2016 Q2")) %>% 
  filter(rec == 1)

Plot the data against the HP trend:

ggplot() +
  geom_rect(aes(xmin = date - 0.5, xmax = date + 0.5, ymin = -Inf, ymax = Inf),
            data = usrec, fill = "gray90") +
  geom_line(data = df, aes(date, hp_trnd), color = "#ca0020") +
  geom_line(data = df, aes(date, emp)) +
  theme_tufte(base_family = "Helvetica") +
  labs(title = "Employment in levels and HP filter", x = NULL, y = NULL,
       subtitle = "1947 Q1-2016 Q2, quarterly", 
       caption = "Note: Gray shaded periods are NBER recessions.") +
  scale_x_continuous(breaks = seq(1950, 2010, by=10))
Nonfarms employment quarterly 1947-2016 and trend from HP filter

So employment is an upwards trending variable and the HP filter (red line) nicely captures that trend.

Next, I implement the new procedure from Hamilton (2017). First, we’ll need some lags of our employment variable. For this, I like to use data.table:

nlags <- 8:11
setDT(df)[, paste0("emp_lag", nlags) := shift(emp, nlags, type = "lag"), ] 

If the true data generating process was a random walk, the optimal prediction would just be its value two years ago. We can calculate this random walk version () right away:

df <- df %>% 
  mutate(cl_noise = emp - emp_lag8)

We want to now regress the current level of the employment variable on lags 8 to 11.

Create the expression for the linear model:

rstr <- paste0("emp ~ ", paste(paste0("emp_lag", nlags), collapse = " + "))

Check it out:

rstr
# [1] "emp ~ emp_lag8 + emp_lag9 + emp_lag10 + emp_lag11"
     

Run the regression:

rmod <- lm(rstr, data = df)

Add fitted values and calculate the cyclical part:

df <- df %>% 
  add_predictions(rmod, var = "trnd_reg") %>% 
  as.tibble() %>% 
  mutate(cl_reg = emp - trnd_reg)

Create a new dataframe that pits the two cyclical components in tidy (“long”) fashion against it other:

cc <- df %>% 
  select(date, cl_noise, cl_reg) %>% 
  gather(var, val, -date) %>% 
  mutate(label = case_when(
    var == "cl_noise" ~ "Random walk",
    var == "cl_reg"   ~ "Regression"))

Which looks like this:

cc

# A tibble: 556 x 4
#   date          var          val label      
#   <S3: yearqtr> <chr>      <dbl> <chr>      
# 1 1947 Q1       cl_noise  NA     Random walk
# 2 1947 Q2       cl_noise  NA     Random walk
# 3 1947 Q3       cl_noise  NA     Random walk
# 4 1947 Q4       cl_noise  NA     Random walk
# 5 1948 Q1       cl_noise  NA     Random walk
# 6 1948 Q2       cl_noise  NA     Random walk
# 7 1948 Q3       cl_noise  NA     Random walk
# 8 1948 Q4       cl_noise  NA     Random walk
# 9 1949 Q1       cl_noise   1.44  Random walk
# 10 1949 Q2       cl_noise  -0.158 Random walk
# ... with 546 more rows
     

Plot the new cycles:

ggplot() +
  geom_rect(aes(xmin = date - 0.5, xmax = date + 0.5, ymin = -Inf, ymax = Inf), 
            data = usrec, fill = "gray90") +
  geom_hline(yintercept = 0, size = 0.4, color = "grey60") +
  geom_line(data = cc, aes(date, val, color = label), size = 0.9) +
  theme_tufte(base_family = "Helvetica") +
  labs(title = "Cyclical employment",
       subtitle = "1947 Q1-2016 Q2, quarterly",
       color = NULL, x = NULL, y = NULL, 
       caption = "See Hamilton (2017) Figure 5.") +
  theme(legend.position = "top") +
  scale_x_continuous(breaks = seq(1950, 2010, by=10))
Cyclical compoment of nonfarms employment using new Hamilton 2017 procedure

This is the lower left panel of Hamilton’s Figure 5.

Let’s check out the moments of the extracted cycles:

cc %>% 
  group_by(var, label) %>% 
  summarise(cl_mn = mean(val, na.rm = TRUE),
            cl_sd = sd(val, na.rm = TRUE))

# A tibble: 2 x 4
# Groups:   var [?]
#  var      label           cl_mn cl_sd
#  <chr>    <chr>           <dbl> <dbl>
# 1 cl_noise Random walk  3.44e+ 0  3.32
# 2 cl_reg   Regression  -4.10e-13  3.09
     

As explained by Hamilton, the noise component has a positive mean, but the regression version is demeaned. The standard deviations exactly match those reported by Hamilton in Table 2.

Let’s compare the HP filtered cycles with the alternative cycles:

ggplot(df, aes(hp_cl, cl_reg)) +
  geom_hline(yintercept = 0, size = 0.3, color = "grey80") +
  geom_vline(xintercept = 0, size = 0.3, color = "grey80") +
  geom_smooth(method = "lm", color = "#ca0020", fill = "#fddbc7", alpha = 0.8,
              size = 0.5) +
  geom_point(stroke = 0, alpha = 0.3, size = 3) +
  theme_tufte(base_family = "Helvetica") +
  labs(title = "Comparing the two detrending methods",
       subtitle = paste0("Correlation: ", 
                         round(cor(df$hp_cl, df$cl_reg, use = "complete.obs"), 2),
                         "."), 
       x = "HP filter", y = "Hamilton 2017 method")
Scatterplot of HP cycles versus Hamilton 2017 cycles

The two series have a correlation of 0.72.

III.

I understand the criticism of the HP filter, but at least everyone knew that it was just a tool you used. You eyeballed your series and tried to extract reasonable trends.

With the new procedure, the smoothing parameter is gone, but instead we have to choose how many years of data to skip to estimate the trend. Hamilton recommends taking a cycle length of two years for business cycle data and five years for the more sluggish credit aggregates. Isn’t that a bit arbitrary, too?

Having to pick whole years as cycle lengths also makes the method quite coarse-grained, as the parameter “years” can only be set to positive integers. Another downside is that the new method truncates the beginning of your sample, because you use that data to estimate the trend. This is a nontrivial problem in macro, where time dimensions are often short.

The Hamilton detrending method has additional benefits, though: It’s backward-looking, while the HP-filter also uses data that’s in the future (but an alternative backward-looking version exists). Also, it adjusts for seasonality by default.

We can only benefit from a better understanding of the problems of the HP filter. And we can always compare results using both methods.

References

Hamilton, James D. (2017). “Why you should never use the Hodrick-Prescott filter.” Review of Economics and Statistics. (doi)