Continuous Optimization is an additional layer which can be used for any type of a test in Nosto. Key functionality is to allocate more traffic for winning variation(s) in the test and less for losing one(s) in order to minimize inevitable performance and revenue cost of a test. A practical example and more about the practical functionality in our product blog here.

Continuous Optimization is enabled in two phases. During first seven days traffic is allocated evenly for all variations, while traffic and performance is analysed. This is called a learning period, which is a mechanism so that the system would not misinterpret the performance of a variation due to potential outliers. 

First optimization can initiate the earliest after 7 days when traffic and performance of variations is analysed, removing potential immediate outliers due to seasonality, traffic peaks or simply too small sample sizes. After 7 days, some exploration & exploitation by reallocating traffic dynamically is allowed, but the system limits extreme traffic allocation by deploying safety parameters in order to allow poorly performing variation(s) to adjust back faster, if it starts to perform better.

Second phase initiates after 14 days when Continuous Optimization can start freely exploring & exploiting all variations. After 14 days, traffic allocated for best performing variation(s) will gradually and potentially dramatically increase and as the counterpoint, decrease for poorly performing variation(s). 

Continuous Optimization analyses the performance and subsequently can reallocate traffic quotas multiple times a day based on automated analysis.