A/X testing

You can include an A/X testing node inside your campaigns to test different communication sequences. These sequences are called variations.

It is possible to create as many variations as you want in the campaign editor. In addition, you can assemble them as you wish so that they contain only one type of message (ex. emails), several channels (ex. email and SMS), the same number of actions, or actions at an odd number. 

A/X testing is relevant when you want to test different sequences of communications or communications with different content.

In a test of this kind, limit the differences you bring to your communications since too many disparities between messages will make it impossible to attribute success to one item. Stick to one differentiator per communication.

This way, you can evaluate different sending sequences to choose only the one that pays the most, based on different selection criteria, such as:

• Click-through/delivery rates
• Open rate
• A low churn rate

A/X Split and A/X Test

You have two A/X testing methods at your disposal:

A/X Split (available only in recurrent campaigns)

This method allows you to create several variations and distribute a percentage of the targeted contacts to them. The sum of the percentages of contacts targeted by the A/X Split node is 100%. During the test period, all contacts entering the node are randomly distributed among the different variations, depending on your configuration. When you select a winning variation, all new contacts entering the node are redirected to that winning variation. Other variations are disabled.

This method is best suited for recurring campaigns, which will be run multiple times over time.

A/X Test (available only in exclusive campaigns)

This method allows you to create several variations and distribute a sample of contacts to them. You can target a fraction of incoming contacts (ex: 20%), while the rest of the contacts (ex: 80%) are put on hold for a winning variation selection. Only the sample of contacts is randomly distributed among the different variations during the test period, depending on your configuration. When you select a winning variation, other contacts on hold are redirected to that winning variation. Other variations are disabled.

This method is best suited for one-time campaigns, which will be run only once.

Create an A/X testing

A/X testing is made up of variations, which correspond to the branches containing the actions you are going to insert. A/X testing allows for two variations or more.

1. Make sure to create the right type of campaign according to the desired testing method: Recurrent campaign for A/X Split and Exclusive campaign for A/X Test.
2. Assemble the beginning of your scenario as you would for a normal campaign, with triggers, delays, conditions, etc.
3. Insert the A/X Split or A/X Test node into your scenario from the Content menu, A/X section.
4. Double-click on the test node to open the editing panel and create variations of your test.
5. Click on the pencil to the right of Variations in the editing panel.
6. In the popup, edit the existing percentages by hovering over the 3 small dots and clicking on the pencil. Modify the proportion using the +/- or by entering the number directly in the box in the percentage zone. You can also add new variations by clicking the Create button. Each variation will have its exit point, where you can connect other nodes.
   
    A/X Split: Automatically, two variations are created of 50% each. Adjust the weight if this 50% allocation does not suit you. Make sure to balance all your variations to get a total of 100%.
   
   A/X Test: Automatically, two variations are created of 10% each. Adjust the weight if this 10% allocation does not suit you. Be sure to balance all your variations so that the assigned weight correspond to the contact sample you want to test. Contacts that are not part of the samples will be put on hold until the winning variation is selected.
   
   
7. Save when the variation setup is completed.
8. In the edit panel, configure the winning variation selection. Two selection modes are available: manual or automated. For more details on these two options, see the Choose the winning variation section below. Note that the automated mode must be configured before activating the campaign.
9. Save when the A/X testing node configuration is completed.
10. Complete your scenario by adding the desired nodes for each variation of your A/X test.
11. Activate your campaign to start testing according to the schedule you programmed in the triggers.

The weight of a variation (percentage) is equivalent to the probability that a contact passes through it compared to the others. Since this is a probability, the number of contacts per variation may not correspond exactly to the percentage, since it is not a case-by-case attribution.

Choose the winning variation

Automated selection

Before activating your campaign, you have the option of setting up an automated selection of the winning variation.

1. In the A/X testing node edit panel, choose Automated selection.
2. Enter the desired date for the selection. For A/X Split, you also have the option to trigger automated selection after several contacts are reached.
3. Select the indicator on which the system will base itself to choose your winning variation.
4. Select the channel(s) on which the system will base itself to calculate the indicators.
5. Save when setup is complete.

Action nodes – Send message must be connected to the A/X testing node output points for the available indicators and channels to display.

When the campaign is activated, it is not possible to modify the configuration of the automated selection. However, you still have the option of making a manual selection if you wish.

When the automated selection is performed by the system, the winning variation will be selected, and the other variations will be deactivated. Only the winning variation branch will process contacts.

Manual selection

When your campaign is activated and has been running for some time, you must return to the interface to choose the winning variation of your A/X testing.

Manual winner selection is very similar between the A/X Split and A/X Test nodes. A/X Test differences are described below.

1. Double-click the A/X testing node in the scenario to open the right editing panel.
2. Click on the trophy to the right of the Selection of winning variation in the edit panel.
3. In the popup, some key information is present:

4. Select your decision indicator.
5. Choose the winning variation.
6. Save when selection is completed.
    At this stage, the process is different for A/X Test. See the section below to find out more.
7. The selected variation will be displayed in blue in the editing panel for reference. Other variations will be disabled. Only the winning variation branch will process contacts.

If contacts have not finished proceeding through the variation that has not been selected, and which then becomes inactive, they will still finish the sequence and will not be stopped. By choosing a winner, it indicates that no new incoming contacts will be able to pass through the inactive branch.

A/X Test

Manual winner selection is very similar between the A/X Split and A/X Test nodes. Following the manual selection description above, here are the differences regarding A/X Test:

6. In the winning variation selection popup, the contacts placed on hold are displayed in the lower-left corner:
7. When your selection is completed, click Continue.
8. In the planning and approval popup, schedule execution for pending contacts. You can choose between Now (following approval), or Later (on a date of your choice).
9. Approve when planning is complete.
10. The selected variation will be displayed in blue in the edit panel for reference. Other variations will be disabled.
11. The start date of the winning variation execution will also be displayed in the editing panel. At runtime, only the winning variation branch will process pending contacts.