Use control groups to measure your campaign

Finally, the power of “what if” is at your fingertips. Learn and witness the measurable impacts your campaigns are having on your audience by using 90/10 control group splits in Lexer Activate. This article will take you through use cases and how to create these splits.

This guide assumes you’ve read through the guide to using audience splits, which goes into detail on what splitting audiences is, and how you can prepare to use audience splits.

Why you would use control group splits?

A Control group split is the division of your audience into a 10% to 90% split. Meaning you’ve got 90% of an audience allocated as your ‘experiment’, and 10% allocated as your ‘control’. But why? Measuring impact. How? By removing 10% (‘control’) of your audience from the advertisement so you can study the natural response of your control group. You can then compare this with your 90% (‘experiment’) and see how many more people by percentage in the experiment (‘90%’) group responded positively when compared to the control (10%). One more time: that’s cutting a small group out from your campaign so you can see how much influence your activation actually had on the 90% when compared to that small 10% who didn’t see it.

Let’s begin.

Step 1: Picking the right Segment

Using your knowledge of Segment, your first task is to build the right Segment and ensure you’ve saved this Segment. Your Segment will require two things:

Rationality; why is it you’re interested in this Segment?
A channel: How is this Segment best reached? Answering these two questions prior will help you know the best way to deliver your campaign effectively prior to using audience splits.

Step 2: Splitting the audience

Now to split our audience and send it over to the appropriate channel. This can be done inside Activate. In Activate, simply configure an audience to be sent to your platform and select the control group split in the activation details.

Create a new activation
Select your channel
Select the Tribe you saved in step 1
Configure the details of your split, and select “Generate Control Group Split”

Your activation will run and notify you when it has reached your channel. Some channels will try and match your audiences to known users of that channel and sometimes wont show your audience until this process is complete. So if you don’t see your split segment arrive straight away, check again shortly.

Step 3: Comparing the results

When you send an audience with Lexer Activate, you can append an audience ID attribute to your customers so you can return to Identify and measure the impact. When splitting an audience we append two audience attributes, Audience (A) and Audience (B). This means you now have the ability to measure the results by audiences individually, or together. When you’ve launched your campaign, you can use other tools within Identify, like Compare, to measure the impact on these customer groups beyond the campaign. To do this, find the “Included in Audience” attribute, find your A and B audiences, and save each as an individual Tribe.

Find “COMPARE” inside Identify.
Select your control and experiment group audiences and generate a comparison.
Use filters to look for most diversity or least diversity.
Click on insights to see the depth of the comparison.

In the image above you can see that the control group has had an impact on total spend, increases the spend of higher value customers when compared to the control group.

Compare instantly shows you the variations of your campaign. Insights to check would be:

how do the order values vary between those who and did not receive the campaign?
what is the engagement rate of the campaign compared to those who didn’t see it? You can do this by including your attributes in your comparison to measure exactly what you want.

To further the insights deeper, you can note how the make up of the segment has evolved. Have any customers entered your high value segments? How do age groups respond to the message, if at all? All of these insights are instantly acquired through exploring your segments in Identify.