How Do Meta A/B Tests Work in Ads Manager?

Meta’s A/B test is a controlled comparison run by Meta rather than by you switching ad sets on and off. You start with a hypothesis, change one part of your setup, and Meta splits the audience so each person can only be shown one version. When the test ends, Meta compares the versions on the metric you picked, such as cost per result. Meta recommends this over informal testing, which it says can produce overlapping audiences and unreliable results.

How does Meta split the audience?

Meta shows each version to a segment of your audience and ensures nobody sees both. Meta says this keeps the audiences evenly split and statistically comparable. Turning ad sets on and off by hand does not do that, and Meta warns it can lead to inefficient delivery and overlapping audiences.

What can you test?

Variable	What you change	Where it’s managed
Creative	Images, text or creative type, such as a video against a carousel	Ad level
Audience	Regions, demographics, or a custom audience against an interest-based one	Ad set level
Placement	Placement types, such as automatic placements against chosen placements	Ad set level
Custom	Several variables at once	Campaign, ad set or ad

Meta recommends starting with a single variable if you are new to A/B testing. With more than one variable, you learn which version won but not which change caused it.

How long should a test run and how much should it spend?

You can run an A/B test for 1 to 30 days. When you set the test budget, Ads Manager shows an estimated power: the likelihood of a statistically significant result given the schedule and budget. Meta recommends at least 80% estimated power. If you duplicate an existing campaign for the test, Meta duplicates its budget too; otherwise the test uses your current ad budgets. Meta recommends giving both versions the same budget for a fair comparison.

How do you create an A/B test in Ads Manager?

1. Go to Ads Manager and click A/B test in the toolbar.
2. Select Make a copy of this ad, or select two existing ads to compare.
3. Choose the ad from the dropdown and click Next.
4. Select the variable to change in the new version, then choose which ad set to copy.
5. Name the test and choose how Meta should determine a winner.
6. Schedule the test’s start and end dates to cover enough time for reliable results.
7. Review all options and publish to begin the test.

Meta also offers a route that starts from an existing ad set or ad: select it, click Duplicate, choose New split test campaign, and pick the variable to change. Meta says this option is being introduced gradually, so you may not have it yet.

How is the winner decided?

You choose the winning metric when you create the test. It can be the campaign’s current result metric, cost per result, or a different specific metric such as cost per purchase. Meta’s A/B testing article also says tests can measure performance on a cost per conversion lift basis. You can create tests in Ads Manager or in the Experiments tool.

FAQ

Can I just turn ad sets on and off to compare them?

Meta advises against it. It says informal testing by turning ad sets or campaigns on and off manually can lead to inefficient ad delivery and unreliable results, partly because audiences can overlap. A Meta A/B test splits the audience so the two groups are evenly divided and statistically comparable.

What is estimated power in a Meta A/B test?

Estimated power is shown when you review the test budget. It is the likelihood that the test reaches a statistically significant result, based on your schedule and budget. Meta recommends running tests with at least 80% estimated power, which may mean a longer schedule or a larger budget.

Can I test more than one change at a time?

Yes, with the custom variable option. Meta notes that when you test more than one variable, you won’t know exactly which variable led to the winning ad set. It suggests multi-variable tests for specific comparisons, such as two products in two locations, not for guiding overall strategy.