Jonathan Hershaff

I have a PhD in economics from the University of Michigan and about ten years of post-graduate experience, six of them in tech. I have worked at Uber, Airbnb, and Stripe, and before that at the Federal Reserve and the Securities and Exchange Commission. I am currently in a full-time senior data science role.

I specialize in causal inference, answering the question “what’s the impact?” of product feature launches, marketing campaigns, or go-to-market strategies. Specifically, I often work on problems where standard experimentation is not feasible, using techniques such as switchback and geo-based experiments, difference-in-differences, synthetic control, and propensity score-based causal models.

Before tech, my research was cited in a winning US Supreme Court case, and I served as an expert economic witness, explaining investment adviser fraud to a grand jury. I have published in peer-reviewed journals and presented at economics and public policy conferences. Most of that work does not come up day to day, but it is where the habit of being careful about evidence came from.

Udacity brought me in to build and teach their introductory causal inference course, covering difference-in-differences, event studies, synthetic control, and regression discontinuity. I wrote the lessons, the code, the practice problems, and the capstone project.

The course is applied rather than theoretical, so someone finishing it can run those methods on real data, use placebo tests to judge whether an estimate holds up, and interpret the coefficients without needing the proofs underneath. That is roughly the bar of an intro applied machine learning course, where you come out able to fit a model, tune it, and evaluate it honestly.

I earn nothing from it, since Udacity paid a fee up front with no residuals, and I am not an affiliate.

Most of what I publish follows the same loop: read the paper or the documentation, simulate data where I control the ground truth, run the method, and report what actually happened. Simulation is the honest way to evaluate a causal method, because it is the only setting where you know the right answer in advance.

That has produced some inconvenient results, which are the ones worth publishing. A marketing mix model whose predictive fit improved while its ROAS estimates got steadily worse. A widely used synthetic control package returning far more false positives than it should. I would rather publish that than another framework post.