Quick answer: A/B test the product pair first, then the price or discount, then the headline and image, and urgency last. Change one thing at a time, keep a holdout group, and pick the winner on profit per eligible view, not on take rate or AOV. If your upsell app has no split-test feature, run a two-weeks-on, two-weeks-off sequential test instead.
Product first. A relevant add-on beats a 20% badge on a random SKU. Then test 10% vs 20% vs no discount. Then one headline. Then the photo. Fake timers last, if ever. Use a holdout so a sale week does not fool you. Shopify Rollouts can split-test theme and checkout configurations on the Grow plan and up, but it does not split two offer prices inside the post-purchase page. Those tests live in your upsell app or in a sequential calendar.
A post-purchase upsell is the one-click offer Shopify shows on a dedicated page after payment and before the Order status (thank-you) page. The buyer already paid. One tap adds the item to the same order. No second checkout.
A/B testing that offer means showing version A to some buyers and version B to others, then comparing the results. Version B might be a different product, a different discount, a different headline, or a different photo.
Three things make this different from testing a product page:
So the scoreboard is not "which version got more yeses." It is "which version made more profit per buyer who actually saw it."
Change one variable per test. If you swap the product, the price, and the headline in the same week, you will not know what worked. This is the queue we use with merchants, from biggest lever to smallest.
Trigger SKU → complement A vs complement B. Example: after a trail shoe, test wool socks vs an insole at the same price. Worksheet: personalize by product. Ideas: best products to offer.
10% vs 20% vs "save $5" vs no discount. Only after the pair is stable. Guide: discount strategy.
One line that names the fit ("Keeps the shoe dry on wet trails") vs a generic "Wait, don't go." Writing help: offer copy.
In-use photo vs plain pack shot. Keep the file small. Most of these views are on phones: mobile thank-you design.
Last. Shopify already places a temporary fulfillment hold on orders in a post-purchase flow, so "add it now or lose it" is half true anyway. Fake countdowns rarely beat a better pair.
Two things that look like tests but are really targeting rules. First-time vs returning buyers should get different offers from the start; split the audiences, then test inside each one: first-time vs returning. One offer vs a chain of offers is its own question: single vs multiple offers. Shopify caps accepts at three per checkout either way.
A holdout is a slice of buyers who see the old offer, or no offer at all. Everyone else sees the new one. You compare the two groups in the same week, so weather, ads, and sales hit both equally.
Without a holdout you are comparing this week to last week. That is not a test. If AOV went up, was it the new offer, or the email you sent on Tuesday?
Most merchants hold out 10–20% of traffic. Small stores sometimes hold out 50% so both groups fill at the same speed.
If your app cannot split traffic, use time as the split: two weeks A, two weeks B. Same days of the week. Never run the "off" weeks over a sale or a launch. That test is about the sale, not the offer.
You need enough eligible views, not enough total orders. If 40% of your checkouts are wallets, a "50/50 split on all orders" is really a split on 60% of them.
A rough rule that holds up: if your take rate is around 10%, you want several hundred views per variant before a 2–3 point gap means anything. A 2-point swing on 80 views is noise. Competitor guides say "wait for 95% significance." Fine, but a significance calculator still needs a few hundred conversions per side to say 95% about a small difference. Most stores doing 300 orders a month do not get there in two weeks.
Ways to get there faster:
We will not hand you a fake "you need 1,000 conversions" rule. Your gap, your variance, your patience. If you cannot wait, you are not testing. You are decorating.
Take rate can lie. AOV can lie. A 20% off variant will often get more taps and less cash after cost of goods. Here is the math for each variant:
That number is the score. Worked example: Variant A, 10% off, 600 views, 54 accepts at $18 net each = $972, minus $8 cost per unit = $540 profit, or $0.90 per view. Variant B, 25% off, 600 views, 78 accepts at $15 net = $1,170, minus $624 cost = $546, or $0.91 per view. B "won" take rate by almost half. It tied on money, and it trained buyers to expect 25% off.
Full worksheet: calculate post-purchase upsell profit. Where each number comes from: track upsell revenue in Shopify, GA4, and Meta.
One warning. Shopify's developer docs say analytics tools on the Pixel API report the purchase value for the initial purchase. If GA4 is your only scoreboard, you may be reading the original checkout, not the add-on. Use the app report and Shopify orders for the test result.
Shopify's native testing tool is Rollouts, under Markets → Rollouts. A launch publishes a change. An experiment shows the change to a percentage of visitors and compares it to the control. Experiments can cover your main theme and your checkout and customer accounts configurations. Shopify's help says experiments need the Grow plan or higher. Docs: Create a rollout.
What that means for upsell tests:
| What you want to test | Rollouts | Upsell app | Sequential |
|---|---|---|---|
| Thank-you block order or layout | Yes, if it lives in a checkout configuration (Grow+) | Sometimes | Yes |
| Offer product A vs B | No | If the app has split tests | Yes |
| 10% vs 20% discount | No | If the app has split tests | Yes |
| Headline or image | No | If the app has split tests | Yes |
| Offer on vs no offer (holdout) | No | Often (show to X% of buyers) | Yes |
| App A vs app B | No | No (one app per store) | Yes, by swapping apps |
Older blogs say "Shopify Plus Experiments." Native split testing is now Rollouts, and it is not Plus-only. But it also does not reach inside the post-purchase page. The one-click offer is a separate slot, and its price and product are app settings, not configuration settings.
Some upsell apps ship built-in split tests with custom traffic percentages and per-variant revenue. Some only let you switch offers. Check the app's own help docs before you buy on the promise of "A/B testing." Roundup: best post-purchase upsell apps.
"After the Trail Shoe, wool socks at full price will make more profit per view than the insole." If you cannot write it in one line, you are testing two things.
Cost of goods, shipping weight, fees, and return rate for both add-ons. You need this before you can name a winner.
Same discount, same headline, same image style, same trigger products. Freeze storewide sales and cart discounts for the test window if you can.
If your app can split traffic, set 50/50 (or keep a 10–20% holdout). If not, plan two weeks per variant on matching weekdays.
Tag them. Ten founder checkouts with 100% accept will make any offer look like a miracle for a day.
Check that the offer renders on a phone on day one. Then leave it alone until the date you wrote down.
Export views, accepts, and accepted revenue per variant from the app. Spot-check five orders in Shopify for the added line. Run the math above.
Product winner found? Next four weeks are the discount test. Do not skip ahead to copy.
Most stores on a simple post-purchase app will not have a 50/50 offer splitter. That is fine. Time is a splitter if you keep the rest of the store still.
Week 0. Freeze ads, theme, and cart discounts if you can. Write the hypothesis. Fill the cost sheet.
Weeks 1–2. Offer A (current pair, current price). Log views, accepts, accepted revenue, refunds on that SKU.
Weeks 3–4. Offer B. Same trigger products. One change only. Same days of the week. If you launch a sitewide 20% sale in week 3, the test is dead. Restart.
Compare profit per eligible view, not raw accepts. A rainy week with fewer orders is not a loss if the rate held. Watch the first-time share too. If week 3 is a TikTok spike of new buyers, you did not test returning customers. More pair ideas: post-purchase upsell examples.
If you later get Rollouts experiments on a thank-you configuration, use that for "offer block above vs below the confirmation," not for 15% vs 20%. Different machines.
Oxify Cart Drawer & Upsell runs the one-click post-purchase offer, thank-you blocks, and the cart drawer from $9.99/mo. Use it to set the offers, the trigger products, and the discounts you are testing, and use its accept report plus Shopify orders as the scoreboard. It is not a statistics suite. Keep the spreadsheet.
Our app. One-click post-purchase offers, thank-you blocks, and a cart drawer with free gifts, BOGO, and volume discounts. From $9.99/mo, 14-day trial. 5.0 from 36 reviews (small sample). Works on every Shopify plan.
Common errors: post-purchase upsell mistakes.
The product pair. If the add-on does not belong with the product they just bought, no headline or discount will save it. Then test the discount, then copy and image, then urgency. Change one thing per test.
Use an upsell app with built-in split testing to show variant A and B to a percentage of buyers each, or run a sequential test: two weeks with offer A, two weeks with offer B, same weekdays, nothing else changed. Score each variant on profit per eligible view.
Not the offer itself. Rollouts experiments (Grow plan and up) can split-test your theme and checkout or accounts configurations, which includes thank-you page block layout. The post-purchase offer's product, price, and copy are app settings, so those tests run in the upsell app or as a sequential test.
A slice of buyers who see no offer, or the old offer, while everyone else sees the new one. It lets you compare both groups in the same week, so seasonality, ads, and sales cannot fool you.
Count eligible views, not orders, because wallet and COD buyers never see the page. With a take rate near 10%, plan on several hundred views per variant before a 2–3 point gap means anything. Small stores should test bigger differences and use sequential tests.
No. A deeper discount usually wins take rate and can lose money. The winner is the variant with the higher profit per eligible view after cost of goods, fees, and returns.
No. Shopify lets you select one post-purchase app at a time under Settings → Checkout → Post-purchase page. To compare apps, run one for two weeks, then the other, and compare profit per view.
Oxify is built to run the offers, not to be a statistics suite. Use it to change the product, discount, and copy you are testing, pair its accept report with Shopify orders, and keep the scoreboard in a spreadsheet.
One change, a holdout, profit per view. That is the whole program. Set up the offers you want to test with Oxify and start the first two-week window today.