Nosto's A/B Testing & Optimization offer a wide range of features and functionalities alowing retailers to test and optimize the impact of various personalization programmes and strategies. 

In this article, we will discover the different test types and methodoligies Nosto supports.

Test types 

Campaign Tests

Campaign tests allows to split traffic of a segment into two or more variations.

Campaign tests supports creating variations using one Nosto Placement. One variation is always represented by one Nosto campaign (recommendation slot or content campaign) in campaign tests. 

For example, variation A consists of one content campaign and variation B consists of one recommendation campaign and are both displayed in one same placement, alternatively depending if one user is allocated to variation A or B.

Experience Tests

Experience tests allows to split traffic of a segment into two or more variations .

Experience tests supports creating variations using one or more Nosto Placements. One variation is always represented by one or more Nosto campaigns (recommendation slot or content campaign) in experience tests. 

For example, variation A consists of one placement used by one content campaign and variation B consists of three placements used buy three content campaigns.

Testing methods 

Split Test 

When creating a campaign or an experience test, you can create your variations manually and explicitly decide what campaign or combination of campaigns is a variation A (base variation), variation B and so forth. 

Multivariate Test 

Multivariate test is only available for Experience Tests as it automates the creation of all possible variations in a test. To do so, there must be a minimum of two placements. 

Example: 

Placement A is selected and Campaign 1 and 2 are eligible in this placement.
Placement B is selected and Campaign 3 and 4 are eligible in this placement. 

Nosto will automatically create the following variations: 

Variation #1: Placement A - Campaign 1 + Placement B - Campaign 3
Variation #2 Placement A - Campaign 1 + Placement B - Campaign 4
Variation #3: Placement A - Campaign 2 + Placement B - Campaign 3
Variation #4: Placement A - Campaign 2 + Placement B - Campaign 4

It would have been possible to create the setup above using a regular split test methodology but you would have needed to manually create these four variations.