How to improve app strategy with A/B testing

How to improve app strategy with A/B testing

A/B testing is a well-known tool used by millions of marketers to improve their websites, optimise conversion rate, make well-informed decisions or simply facilitate senior stakeholders approval. Today, it’s also become very helpful for mobile app testing.

Here at Aquafadas, with our Next Gen App it’s possible to run varied A/B apps tests. Options are almost endless: best components, CTAs, messages, headlines, forms – practically everything embedded into HTML app frame. It’s even possible to test by location comparing different designs by country or region.

How do I do it?

At Aquafadas we’ve been using Visual Website Optimizer but the steps you will need to take will be exactly the same if you choose to use Optimizely, AB testy or any other A/B test software.

Check the below scenario written by our Head of Marketing, Cédric Malet, based on the inspiration app: Aquafadas Wave, to see how at Aquafadas we test different headlines and CTAs.

Step 1. Define what you want to test

For this blog, we decided to use our own app – Aquafadas Wave.

But firstly let me tell you what it’s about and why we decided to build it.

  • Source of inspiration – a few years ago, it was hard for us to demo our latest enrichments. Images don’t show their full potential, GIFs have their own limitations and video creation is time consuming.
  • Aquafadas ambassadors showcase – we have an engaged community of talented designers, but how do we help them promote their work and inspire each other?
  • Helping hand for sales – they need to engage and convert prospects with great content and show the power of our solutions. But it was difficult for them to demo a variety of apps in a short period of time.

Our solution was simple and of course app related ;)! We decide to merge all our great content (zave file) into an app. And that’s how and why our Aquafadas Wave app was born. But to make it succeed we needed to ensure that our users share their best content with us and therefore this is what we had to optimize the most! For now, the Wave app users are encouraged to share their content from the component below, directly linked to a web form.


Ab testing step 1

Step 2. Come up with A B scenarios to compare

The dilemma we had to solve was tricky. We needed appealing headlines and well-integrated CTAs. But how?  Through pushy and shouty titles? Or maybe big banners, prominent CTAs action – all in red and bold? Or perhaps less is more and an engaging app component will be sufficient?

Version control:

At besting version control

Version A:

Ab testing version A

Version B:


Ab testing version b

Step 3. Setup and run your test

Now you have to login into your usual A/B test software (in our case, then create a split URL campaign.

Split URL test campaign helps you to test between multiple versions of your component hosted on different URLs. Thanks to split URL testing you can distribute traffic between different URLs of the same component. What is great, you don’t need to change the component URL in Aquafadas management panel Cloud Connect. The A/B test software will automatically send your traffic to different URL.


Follow these steps to create a split URL test campaign:

  1. Log in to VWO and click Create and select Website Campaigns option
  2. In the Select Campaign Type page, click Split URL Test option. The campaign URLs page displays as following
  3. Enter page urls to test
  4. Select a URL match type and enter the URL of the control page. Select a URL match type and enter the URL of the variation page
  5. Click Add Another Variation to include more variation URLs
  6. Enter URL of the page to create previews for the campaign. This URL is not used in the actual campaign, but is used to create preview screenshots for the campaign
  7. Select Enable campaign for a specific visitor group option to target the campaign to a specific visitor segment.
  8. Enable Exclude URLs if you want to exclude certain URLs from the campaign. You can add multiple URLs to exclude from the campaign
  9. Click Next to define conversion goals for the campaign
  10. Setup goal conversion
  11. In the Conversion Goals page, define goals or events you want to track during the campaign. You can add multiple goals to track for the campaign or simply track the engagement.

Step 4.  Analyze test’s results

Now it’s time to review your data. Below are our Wave app’s results.

In the test, conversions mean clicks on the component. Then the component is redirecting users to the web form.

Data results

It was a good test – we are pleased! The version control is the worst version we tested! We can only do better. We can see that the version A and B got almost the same results. A is just a little bit better. However, if we compare version A to the original version (control) we see that we increased the traffic to the form by 28%! It’s a big improvement without spending too much time on it. What a great result!

Now it’s your time to try. Give it a go and let us know whether it helped you make these important app strategy decisions.