Comparing creatives using an A/B split

Never before have simple marketing tools been so powerful. With the power of Identify at your fingertips, you’re about to turn around campaign insights on an A/B split in record time - and this guide is going to show you how.

This guide assumes you’ve read through the guide to using audience splits , which goes into detail on what splitting audiences is, and how you can prepare to use audience splits.

Let’s get started.

Throughout this article, there are references to the following two pieces of example creative. On the left is Creative A, and on the right is Creative B. Your goal is to find out which is going to more effective at capturing your audience’s attention.

To help you learn which creative is more effective, we need to divide our audience evenly, and without bias in Lexer. To do this we first need to pick which segment we want to target using Identify.

Step 1: Picking the right segment.

Using your knowledge of Identify, your first task is to build the right segment and ensure you’ve saved this segment as a Tribe. Your segment will require two things:

  1. Rationality; why is it you’re interested in this segment?
  2. A channel: How is this segment best reached? Answering these two questions prior will help you know the best way to deliver your campaign effectively prior to using audience splits.
  1. Build your audience, here we’re finding ecom customer’s who are interested in coffee
  2. Review the attributes pivot to grow or shrink your audience with customer insight
  3. Once happy with your search, save your Tribe

Step 2: Splitting the segment.

Now to split our audience and send it over to the appropriate email platform. This can be done inside Activate. In Activate, simply configure an audience to be sent to your platform and elect the A/B split in the activation details.

  1. Create a new activation
  2. Select your channel
  3. Select the segment you saved in step 1
  4. Configure the details of your split, and select “Generate A/B Split”

Your activation will run and notify you when it has reached your channel. Some channels will try and match your audiences to known users of that channel and sometimes wont show your audience until this process is complete. So if you don’t see your split segment arrive straight away, check again shortly.

Step 3: Comparing the results.

When you send an audience with Lexer Activate, you can append an audience ID attribute to your customers so you can return to Identify and measure the impact. When splitting an audience we append two audience attributes, Audience (A) and Audience (B). This means you now have the ability to measure the results by audiences individually, or together. When you’ve launched your campaign, you can use other tools within Identify, like Compare, to measure the impact on these customer groups beyond the campaign. To do this, find the “Included in Audience” attribute, find your A and B audiences, and save each as an individual Tribe.

  1. Find “COMPARE” inside Identify
  2. Select your A/B Audiences and generate a comparison
  3. Use filters to look for most diversity or least diversity
  4. Click on insights to see the depth of the comparison

In the results above, we can see there’s a 1.6x decrease in website visits, but an increase in higher value purchases from customers in the blue advertisement bucket. This is likely because of the time they’ve spent in-store browsing the products and enjoying a warm coffee.

Compare instantly shows you the variations of the impact of your campaign. Insights to check would be

  • how do the order values vary?
  • what is the engagement rate of the campaign comparatively?
  • how many customers from each converted into in-store purchases? You can do this by including your attributes in your comparison to measure exactly what you want. To learn more about how Tribe Comparison works, click here.

To further the insights deeper, you can note how the makeup of the segment has evolved. Have any customers entered your high-value segments? How do age groups respond to either message? All of these insights are instantly acquired through exploring your segments in Identify.

Further reading

For more insights into how you can use splits to fuel your marketing power, read our article on control group splits.