Writing
Simulations, mostly. I read the paper or the documentation, generate data where I control the ground truth, run the method, and write up what actually happened, including when the result is inconvenient.
The Rollout Trap: Why a Positive A/B Test Doesn’t Mean Treat Everyone
A positive average treatment effect does not tell you who should get the change. I simulated 100,000 searchers, trained a T-learner on the experiment data alone, and checked the targeting decision against ground truth I built myself.
Read the write-up →
The Waiting Tax: How a Surrogate Index Answers a 12-Month Question in 4
The metric you care about takes a year to observe, and nobody wants to hold a decision that long. I simulated a subscription business where I knew the true 12-month answer, then checked how early a surrogate index could recover it.
Read the write-up →
The Linear Trap: Why Double Machine Learning Beats Fixed Effects
I built a simulation to stress-test whether Double Machine Learning actually changes decisions compared to fixed effects OLS. It does, and the gap shows up exactly where product experiences stop being linear.
Read the write-up →
The False Positive Trap: The Dark Side of Bayesian A/B Testing
If you use a history of winning features to boost power, what happens when you test a bad idea? With ten historical winners and no discount, the false positive rate hits 75%.
Read the write-up →
Surviving the “Haircut”: Why Bayesian A/B Testing Beats Conservatism
Stakeholders rarely accept historical lifts at face value. I ran 1,000 simulations to find out how much of a conservative discount a Bayesian prior can absorb before it loses its power advantage.
Read the write-up →
Don’t Trust the Closed-Form: Sizing Switchback Experiments with Permutation Tests
Standard power calculators assume your observations are independent. Time-series data is not, and on a two-week switchback the closed-form standard error understated the true variance by 22%.
Read the write-up →
Slashing Amazon Ad Spend by 69%: Automating a Switchback Experiment to Find the Optimal Bid
Standard A/B testing is impossible in walled gardens. Here is how I built an automated switchback pipeline on my own seller account to bypass platform attribution, cut ad spend by 69%, and increase net profit.
Read the write-up →
The MAPE Illusion in Marketing Mix Modeling
Why a Better Fitting Model Doesn’t Mean Better Attribution In Data Science, predictive fit metrics like Mean Absolute Percentage Error (MAPE) or R² are our compass. It is highly intuitive—and common practice—to assume that…
Read the write-up →
Data Science Case Study Interviews: The One Thing Most Candidates Skip
Technically strong candidates fail case studies by rushing past product sense. Here is what the strong ones do in the first few minutes instead.
Read the write-up →
Estimating Amazon Ad Incrementality for Sellers: Combining Switchback Tests with MMM
I don't have access to Amazon Marketing Cloud. Here is how I used a manual switchback experiment and Google Meridian to triangulate the true value of my ad spend, and why the platform number was almost double the truth.
Read the write-up →