A/B Splits

Discover which of your campaigns is more effective using A/B splits

Never before have simple marketing tools been so powerful. With the power of Segment at your fingertips, you’re about to turn around campaign insights on an A/B split in record time - and this guide is going to show you how.

This guide assumes you’ve read through the guide to using audience splits, which goes into detail on what splitting audiences is, and how you can prepare to use audience splits.

Let’s get started.

Throughout this article, there are references to the following two pieces of example creative. On the left is Creative A, and on the right is Creative B. Your goal is to find out which is going to more effective at capturing your audience’s attention.

To help you learn which creative is more effective, we need to divide our audience evenly, and without bias in Lexer. To do this we first need to pick which segment we want to target using Segment.

Step 1: Picking the right Segment.

Your first task is to build the right Segment and ensure you’ve saved this. Your Segment will require two things:

  1. Rationality; why is it you’re interested in this segment?
  2. A channel: How is this segment best reached? Answering these two questions prior will help you know the best way to deliver your campaign effectively prior to using audience splits.

  1. Build your audience, here we’re finding ecom customer’s who are interested in coffee
  2. Review the attributes pivot to grow or shrink your audience with customer insight
  3. Once happy with your search, save your Segment

Step 2: Splitting the segment.

Now to split our audience and send it over to the appropriate email platform. This can be done inside Activate. In Activate, simply configure an audience to be sent to your platform and elect the A/B split in the activation details.

  1. Create a new activation
  2. Select your channel
  3. Select the segment you saved in step 1
  4. Configure the details of your split, and select “Generate A/B Split”

Your activation will run and notify you when it has reached your channel. Some channels will try and match your audiences to known users of that channel and sometimes wont show your audience until this process is complete. So if you don’t see your split segment arrive straight away, check again shortly.

Step 3: Comparing the results.

When you send an audience with Lexer Activate, you can append an audience ID attribute to your customers so you can return to Segment and measure the impact. When splitting an audience we append two audience attributes, Audience (A) and Audience (B). This means you now have the ability to measure the results by audiences individually, or together. When you’ve launched your campaign, you can use other tools within Understand, like Compare, to measure the impact on these customer groups beyond the campaign. To do this, find the “Included in Audience” attribute, find your A and B audiences, and save each as an individual Segment.

  1. Find Compare inside Understand
  2. Select your A/B Audiences and generate a comparison
  3. Use filters to look for most diversity or least diversity
  4. Click on insights to see the depth of the comparison

In the results above, we can see there’s a 1.6x decrease in website visits, but an increase in higher value purchases from customers in the blue advertisement bucket. This is likely because of the time they’ve spent in-store browsing the products and enjoying a warm coffee.

Compare instantly shows you the variations of the impact of your campaign. Insights to check would be

  • how do the order values vary?
  • what is the engagement rate of the campaign comparatively?
  • how many customers from each converted into in-store purchases? You can do this by including your attributes in your comparison to measure exactly what you want.

To further the insights deeper, you can note how the makeup of the segment has evolved. Have any customers entered your high-value segments? How do age groups respond to either message? All of these insights are instantly acquired through exploring your segments in Segment.

Powered by HelpDocs (opens in a new tab)