Nosto's A/B Testing & Optimization offer a wide range of features and functionalities alowing retailers to test and optimize the impact of various personalization programmes and strategies. 

In this article, we will discover the different test types and methodoligies Nosto supports.

Test types 

Campaign Tests

Campaign tests allows to split traffic of a segment into two or more variations.

Campaign tests supports creating variations using one Nosto Placement. One variation is always represented by one Nosto campaign (recommendation slot or content campaign) in campaign tests. 

For example, variation A consists of one content campaign and variation B consists of one recommendation campaign and are both displayed in one same placement, alternatively depending if one user is allocated to variation A or B.

Experience Tests

Experience tests allows to split traffic of a segment into two or more variations .

Experience tests supports creating variations using one or more Nosto Placements. One variation is always represented by one or more Nosto campaigns (recommendation slot or content campaign) in experience tests. 

For example, variation A consists of one placement used by one content campaign and variation B consists of three placements used buy three content campaigns.

Sequence Test

It allows you to test a Category Merchandising sequence against multiple variations

Split Test 

When creating a campaign or an experience test, you can create your variations manually and explicitly decide what campaign or combination of campaigns is a variation A (base variation), variation B and so forth. 

Category Merchandising Test

This type of testing focuses on understanding how different merchandising rules affect performance on specific category pages or across all categories. It allows you to compare different variations of merchandising rules

Query Merchandising Test

Similar to category merchandising, query merchandising tests focus on search queries rather than categories. This allows you to test how different merchandising rules affect the performance of search results for specific queries or across all search terms.

Multivariate Test 

Multivariate test is only available for Experience Tests as it automates the creation of all possible variations in a test. To do so, there must be a minimum of two placements. 

Example: 

Placement A is selected and Campaign 1 and 2 are eligible in this placement.
Placement B is selected and Campaign 3 and 4 are eligible in this placement. 

Nosto will automatically create the following variations: 

Variation #1: Placement A - Campaign 1 + Placement B - Campaign 3
Variation #2 Placement A - Campaign 1 + Placement B - Campaign 4
Variation #3: Placement A - Campaign 2 + Placement B - Campaign 3
Variation #4: Placement A - Campaign 2 + Placement B - Campaign 4

It would have been possible to create the setup above using a regular split test methodology but you would have needed to manually create these four variations.