Search
Close this search box.

Andre Kempe,

CEO,

Admiral Media,

Jan 10, 2023

Why Apple Custom Product Pages are NOT A/B tests: Here’s what you need to know

In 2022, Apple introduced Custom Product Pages (CPP), which replaced the previous limited Creative Sets for Apple Search Ads. Before this, the App Store was basically a static storefront, meaning that all users in a specific location saw the same App Store page for a given app. Marketers had limited things to work with to test which version of their product page would be most effective. Third-party tools could mimic the App Store and give an idea, but they weren’t the real deal because it’s not testing on the platform. But why would marketers want the ability to customize? There are two main reasons that require having variations of an App Store page:

  1. A/B testing two variants of a page to keep what works and improve from there, Google has been offering this functionality on Android for years.
  2. Positioning the product differently to specific audiences that can help increase conversions and cast a wider net.

Two types of App Store page variations: PPO and CPPs

With iOS 15, Apple changed its ways and started to offer Product Page Optimization (PPO) for A/B testing up to 3 variants using Bayesian statistical analysis. Though PPO is a step up from what existed before, they also are quite limited and can only be used on the default App Store page.

Why Apple Custom Product Pages are NOT A/B tests: Here's what you need to know Product Page Optimization (PPO) can be used to test page elements such as icons, screenshots, and App Previews

Why Apple Custom Product Pages are NOT A/B tests: Here's what you need to know This means that they can’t be combined with Custom Product Pages (CPP) which are aimed at the use case 2) different positioning for different audiences. Custom Product Pages are built to be used in paid promotion activities, such as on Apple Search Ads, and more recently, Facebook Ads. Apple allows for up to 35 different variants. This is especially interesting to apps that have broad, diverse content. Let’s take for example an app that offers different kinds of classes – painting classes, DIY, and boxing. Such an app could use custom pages to attract these likely very different audience segments at the same time.

CPPs let you address different audiences at once

This use case is also great for an app like our client ImmoScout24, a leading German marketplace app, who are serving two distinct custom groups: renters and buyers of houses and apartments. Even though both of these user groups are looking for apartments, there is a solid hypothesis that these two groups have little overlap, different needs, and different customer lifecycles. Generating and testing Custom Product Pages showed us that this was true and resulted in a substantial performance boost. You can read more about the award-winning case study here, but to summarize: We were able to reduce the CPA by half compared to the default page by using Custom Product Pages for the buyer and renter segments.  ImmoScout24 Case Study by Admiral Media Another great example from our client Fastic, converting gender-specific messaging through CPPs:

Custom Product Pages - Fastic by Admiral Media

Fastic uses CPPs to target women and men, highlighting different benefits

Why CPPs are not A/B tests 

While CPPs are a great and versatile tool, there are some cautions, though, which we will cover here. Unlike PPO, Apple’s Custom Product Pages are not A/B tests, but why does it matter? A/B testing is designed to compare multiple versions of a single variable, so to A/B test an App Store and determine which one is better, you have to make sure that all other variables are identical, or close to it, to compare results. The two different pages should be shown to the same statistically relevant number of users, from the same audience, and the same devices, at the same time. Only then can you be certain that a positive effect is caused by the version of the App Store page. Custom Product Pages are only shown to users on iOS 15.2 or higher (which is when the feature was released). However, it’s uncertain who the default page is shown.

That’s why app growth expert Thomas Petit recommends not to jump to conclusions comparing CPP to default page performance: “Custom Product Pages CPP is not an A/B test: you can’t compare Default vs. CPP because those are not shown to the same audience: 100% of iOS15.2 see the CPP, and 100% of iOS <15.1 users see the default page.” With so many variables being different, Thomas is not satisfied with attributing any uplift to a custom page without taking into account any of the other attributes that were different (e.g. newer OS version, audience selection). Thomas Petit: “The only way to compare is to a before/after the test.” But it’s still an approximation to get closer to an unbiased result.

A before/after the test is also currently the best practice to compare CPPs against each other because they need to collect enough data to be significant, which takes time. Apple’s algorithm won’t distribute the audiences evenly when running CPPs against each other in different ad groups at the same time. With this being said, Thomas agrees about the value that Custom Pages have when using them for the use case 2) positioning an app differently to attract specific audiences. Thomas Petit: “My general stance is that on paper, CPP can totally be a big win (overall, and for Apple Search Ads), when there’s a specific context. However, if it’s an undefined intent context, one can’t expect too much uplift.”

CPPs – a lot of potentials but still early 

Since October 2022, Facebook Ads also offers advertisers the option to use CPPs in their campaigns. We jumped on it and tested Custom Product Pages with Facebook Ads right away, but our initial results were inconclusive. One combination of ad and CPP led to a 20% higher conversion rate and thus lower CPIs. For others, the results were mostly similar, or sometimes worse, compared to the default page.

Custom Product Pages are still early and will likely improve over time, especially when they will be more equally distributed to users in the future. As of today, they’re an exciting opportunity to promote specific features or to differentiate audiences for an app. However, they’re not an automatic win ‘just because’ they’re customized. They need a specific use case, a strong hypothesis, and a thorough and realistic result assessment. With this in mind, we saw that CPP can already deliver real wins, and we’ll keep on testing them on ASA and Facebook to find the next winners.

 

If you’re interested in ways you can measure Apple CPPs, apart from App Store Connect, check out our blog post How to Measure Apple CPP.

Join +3.000 app marketers and beat your competitors

YOU MAY ALSO LIKE