# CUPED: The Formula, a Worked Example and When It Fails

> CUPED uses pre-experiment data to cut A/B test variance. See the formula, a worked example, and when it doesn't pay off, with primary-source numbers.

- URL: https://missiongrowth.io/blog/cuped
- Published: 2026-08-16 · Updated: 2026-09-24
- Author: Ömer Furkan Aktaş, Founder, Mission Growth
- Publisher: Mission Growth. Company facts: https://missiongrowth.io/llms.txt

A vendor pitch says CUPED will make your A/B tests "up to 60% faster." Another says 30-50%. A third says it's akin to 20% more traffic on the same test.

You've seen the term work its way into a growth or product conversation, and you don't yet know if any of those numbers apply to the metric you're actually testing.

They don't apply as stated. Growth teams increasingly treat CUPED A/B testing as a single switch to flip, without checking whether their metric even qualifies. Your CUPED variance reduction is the square of the correlation between your metric's pre- and post-experiment values. A vendor's headline percentage describes its own case study, and yours is different.

In this guide:
- The CUPED formula, and why correlation is the whole story behind it
- A hand-worked example showing exactly what the adjustment does to real numbers
- Which metrics actually benefit, and which barely move
- When CUPED fails, and what to check before you trust the result

## What CUPED is, and why "up to 60% faster" oversells it

Microsoft researchers introduced CUPED, short for Controlled-experiment Using Pre-Experiment Data, in a 2013 paper.

It's a statistical adjustment that uses a user's own pre-experiment behavior to strip predictable noise out of an A/B test's outcome metric. The CUPED meaning is exactly what its acronym spells out: it controls an experiment using data collected before that experiment started.

CUPED needs two things to work:

- Pre-experiment data for the same users
- A metric that correlates with its own past values

It doesn't guarantee a fixed speedup, and it doesn't help every metric by the same amount.

That's the part vendor pages skip. Optimizely's own page states CUPED as "potentially cutting sample size requirements by 30-50% for metrics with a strong correlation to historical behavior." Kameleoon reports "up to 60%" reduction in required sample size, from correlating one metric (main-goal conversion) across more than 200 of its own experiments.

Both numbers are real. Neither is universal: each holds for the specific metric that vendor measured, not for whatever you plug in next.

GrowthBook's documentation goes further, attributing a bigger number to Microsoft itself:

> ...akin to adding 20% more traffic...

That claim traces to a November 2022 Microsoft article. Read the article directly and the figure isn't there.

What the article actually reports, quoted from its own figure caption: "From a simulated total R² of 0.4, the median effective traffic multiplier is 1.66 in simulations. This translates to a power gain of 22%." That's one simulated scenario at one correlation strength, not a general "20% more traffic" finding, and Microsoft's own page never states it that way.

So why it matters: none of these numbers tell you what CUPED will do for your metric. What decides that is one variable, covered next.

## The CUPED formula, and why correlation is the whole story

CUPED adjusts a user's post-experiment metric by subtracting a scaled version of their pre-experiment value: Ŷcv = Y − θX + θE[X].

- θ = Cov(Y,X)/Var(X): the single number that minimizes the adjusted metric's variance
- Y is the metric you're testing; X is that same user's value on a correlated metric measured before the experiment started

θ tells you how much of X's variation to subtract back out. Deng, Xu, Kohavi and Walker set this out in the original CUPED paper, a 2013 WSDM publication.

The consequence is what matters here. The resulting variance is var(Y)(1 − ρ²), where ρ is the correlation between a user's pre- and post-period values. The size of the gain is fixed the moment you pick your covariate: it's the square of its correlation with the outcome.

That fraction does double duty. Statistical power depends on variance divided by sample size, so a variance cut sized to ρ² is mathematically identical to needing only (1 − ρ²) times the original sample size, or duration, for the same power.

That's the exact arithmetic behind the paper's own headline line: CUPED gets you the same statistical power "with about only half the users, or half the duration." Half comes from a correlation whose square lands near one-half, which is exactly what Bing's own three-experiment average implies (see below).

## A worked example: adjusting a metric by hand

CUPED's adjustment becomes mechanical once you see it applied to real numbers.

Here's a CUPED example built from small, invented numbers, chosen only to make the mechanics easy to follow by hand; they're not Mission Growth or client data.

Picture five users in a test, each with a pre-period value (X) and a post-period value (Y) on the metric you're measuring. In this group the pre/post covariance is 1.5 and the pre-period variance is 2.5, so theta is 1.5 divided by 2.5, or 0.6.

The group's average pre-period value is 4. Each user's adjusted value is their raw Y minus 0.6 times the distance between their own X and that average.

::dataset{key="cuped-worked-example" name="Illustrative CUPED adjustment example (invented numbers), September 2026"}

| User | Pre-period value (X) | Post-period value (Y) | Adjusted value |
|---|---|---|---|
| 1 | 2 | 5 | 6.2 |
| 2 | 4 | 7 | 7 |
| 3 | 6 | 8 | 6.8 |
| 4 | 3 | 6 | 6.6 |
| 5 | 5 | 6 | 5.4 |

Add up the raw column and the adjusted column and they land on the same total. That's the proof CUPED doesn't manufacture a result: it only redistributes each user's outcome around the shared average.

The variance of the outcome falls from 1.3 to 0.4 in this table, so the standard error shrinks. The estimated lift doesn't move.

## Which metrics actually benefit from CUPED

CUPED benefits a metric in direct proportion to how strongly that metric correlates with its own pre-experiment value, because the variance cut equals that correlation squared.

Bing's own three most recently reported experiments cut variance by 45%, 52% and 49%, an average of 48.7% (computed by averaging the three published figures).

Run that 48.7% back through the formula, where reduction equals ρ², and the implied correlation is about 0.70, since 0.70² ≈ 0.487.

Netflix's case study (Xie and Aurisset, KDD 2016) found the same relationship from the other direction. For new users, the pre- and post-period correlation runs only 0.2 to 0.4, and reported variance reduction is "very low regardless of the metric or the variance reduction technique used."

Run that correlation range through the same formula and the predicted reduction is 4% to 16%, well below Bing's headline number. Bing's own revenue-per-user metric tells the same story from a third angle: an exception where correlation is weak, and variance drops by less than 5%.

| Metric / correlation | Reported or implied correlation | Variance reduction |
|---|---|---|
| Bing average, three experiments | about 0.70 | 48.7% |
| New users (Netflix) | 0.2 to 0.4 | 4% to 16% (predicted) |
| Revenue-per-user (Bing) | low | under 5% |

The table restates a simple rule: check your own metric's pre/post correlation before you promise a stakeholder a speedup, because the same technique that roughly halves one team's test duration barely moves another's.

::figure{src="/blog/figures/cuped-1.svg" alt="CUPED variance reduction by pre/post correlation: 48.7% at ~0.70 correlation, 4-16% for new users, under 5% for revenue-per-user." caption="The same technique cuts one metric’s noise in half and barely touches another, and the difference is entirely the pre/post correlation." width="720" height="279"}

This is the same logic behind [minimum detectable effect](https://missiongrowth.io/blog/minimum-detectable-effect): a lower MDE at the same traffic comes from that same variance cut, the one lever driving both.

The same correlation logic applies beyond product experiments, too. A metric that varies widely by campaign type, like [SEO ROI](https://missiongrowth.io/blog/seo-roi), is exactly the kind that benefits from checking your own pre/post correlation before you commit traffic to a holdout test, instead of trusting a single reported percentage across every campaign type.

## Choosing your covariate and pre-experiment window

CUPED works best when its covariate is the pre-experiment version of the exact metric you're testing, beating alternatives like a user's entry day by a wide margin. Bing's own comparison, run directly on queries per user, makes the gap concrete:

- **Same metric, pre-period value alone:** more than 45% variance reduction.
- **Entry day alone:** about 9-10% variance reduction.
- **Both combined:** only 2-3 percentage points more than the pre-period metric by itself.

::figure{src="/blog/figures/cuped-3.svg" alt="A user’s own pre-period value as a CUPED covariate cuts variance more than 45%, compared to about 9-10% for entry day alone." caption="Entry day alone captures only a small share of the variance cut a user’s own pre-period value delivers." width="720" height="182"}

Stacking a second, weaker covariate barely helps once you already have the strong one. The same paper recommends 1-2 weeks of pre-experiment data: shorter periods match users poorly, and longer ones start to lose their correlation with what's happening in the current experiment period.

That window has to fit inside whatever cadence you already run tests on. A covariate window that runs longer than your usual [growth experiment cadence](https://missiongrowth.io/blog/growth-experiment-cadence) fights your own planning rhythm instead of fitting it.

Some platforms build variants of this into their product. GrowthBook's own CUPEDps feature is one example, adjusting for stratification across a cohort rather than a single covariate. It's a platform-specific configuration, not something every reader needs to evaluate.

## When CUPED doesn't pay off

CUPED does the least good for new users, since even a proxy covariate correlates only 0.2 to 0.4 with their outcome.

It does the most damage when randomization was never clean to begin with. A first-time visitor has no pre-experiment history for CUPED to adjust with.

Netflix's own case study found that for new users, variance reduction is "very low regardless of the metric or the variance reduction technique used," which rules out swapping in a different adjustment method as the fix.

The second failure mode is harder to spot because it's invisible in the output. CUPED assumes the pre-period average is the same in treatment and control before the experiment starts. Randomization is supposed to guarantee that automatically.

When it doesn't, CUPED's adjustment stops being neutral, and the decision rule in the next section covers exactly what to trust instead. Before you act on a CUPED-adjusted result, run a [sample ratio mismatch](https://missiongrowth.io/blog/sample-ratio-mismatch) check first: CUPED only reports a cleaner number, it never checks whether the split behind that number was clean to begin with.

## CUPED vs. difference-in-differences and autoregression

CUPED reaches the same answer as difference-in-differences and autoregression under clean randomization. The moment randomization is imperfect, CUPED alone turns biased while the other two do not.

The primary paper's own unbiasedness condition explains why: CUPED's adjustment stays unbiased only when the pre-period average is the same in treatment and control, a condition clean randomization guarantees automatically, before any treatment effect exists. Break that condition and CUPED inherits the imbalance instead of correcting for it.

A 2024 Monte Carlo simulation shows the size of the problem with real numbers. Against a true effect of 2, CUPED's average estimate under imperfect randomization came out around 1.58 (biased), while difference-in-differences and autoregression both averaged about 1.99 (unbiased). Autoregression reaches the same unbiased answer as difference-in-differences, just less efficiently.

| Condition | Estimator to trust |
|---|---|
| Randomization confirmed clean | CUPED (fastest, most efficient) |
| Randomization in question or confirmed imperfect | Difference-in-differences or autoregression |

That's the one decision point worth checking before you reach for CUPED by default: it's the fastest option, and safety depends on confirming clean randomization first. Once you've worked out your own pre/post numbers, run the numbers through [our SEO ROI calculator](https://missiongrowth.io/tools/seo-roi-calculator) to see the before-and-after difference for yourself instead of trusting a single adjusted figure.

::figure{src="/blog/figures/cuped-2.svg" alt="CUPED, difference-in-differences and autoregression compared across clean and imperfect randomization: which stay unbiased." caption="Difference-in-differences and autoregression stay unbiased when randomization breaks; CUPED alone does not." width="720" height="379"}

## FAQ

### Can you combine CUPED with sequential testing?
Yes. They solve different problems, variance reduction versus valid early stopping, and teams commonly run both together. The peeking mechanics that make early stopping valid are a separate question; see [sequential testing](https://missiongrowth.io/blog/sequential-testing) for how to do that safely.

### What if I have no pre-experiment data for a user?
CUPED can't adjust that user's result. The original paper's own fix is a binary covariate (seen in the pre-period or not), but that still leaves genuinely new users unadjusted, since a yes/no flag carries almost none of the correlation a real pre-period value would.

### What does CUPED stand for?
Controlled-experiment Using Pre-Experiment Data. The name describes exactly what it does: it uses data from before your experiment started to control for noise that would otherwise show up in the result.

### Is CUPED the same as regression adjustment?
Closely related but not identical. CUPED with θ fixed at its optimal value is a special case of regression adjustment. The two diverge once randomization is imperfect: regression-based methods like difference-in-differences and autoregression stay unbiased where CUPED alone does not.

### How much pre-experiment data do you need?
About 1-2 weeks, based on Bing's own testing. Shorter windows match users poorly; longer ones lose their correlation with what's happening in the current experiment period, which cancels out the extra history.

### Should you check for a sample ratio mismatch before trusting a CUPED result?
Yes. CUPED assumes clean randomization and doesn't verify it. A [sample ratio mismatch check](https://missiongrowth.io/blog/sample-ratio-mismatch) confirms the assumption CUPED's whole adjustment depends on, before you act on the number it gives you.

CUPED's payoff was never a vendor's flat percentage. It's the square of your metric's pre/post correlation, the same math that cut Bing's own average test duration by close to half and barely moved Netflix's new-user metrics.

Before you promise a stakeholder a speedup, pull your own metric's pre- and post-period values and compute that correlation by hand. If it's high, CUPED will do real work. If it's not, the formula will tell you that too, before you spend the traffic finding out.