All Collections
A/B Testing & Optimization FAQ
When shall I analyze the results of my tests?
When shall I analyze the results of my tests?
Lari Lehtonen avatar
Written by Lari Lehtonen
Updated over a week ago

As a rule of thumb, it’s recommended to wait for a minimum period of time of 14 days to wait before analyzing the results of a test. All Nosto tests have a Data Gathering... state during the first 14 days after activation. While you can peak at the results of the variations during those 14 days, it's recommended not to end a test during that data gathering period.

Nosto shares the same positioning than Google, that expresses it clearly in the following statement: 

Why two weeks?

With tests designed to be deployed to regularly-encountered parts of a digital property, that’s generally a good period of time to get a well-rounded sense of your data, including weekdays, weekends, and any other anomalies that might happen from one week to the next. However, two weeks is a minimum, and experiments can run longer. You can also make a call earlier if you think your traffic isn’t likely to change qualitatively, although we don’t recommend doing this.

If you do decide to leave a test live for a longer period of two weeks, it's recommended to respect the full weekly cycles to avoid misleading results. We recommend you the following articles here and here covering extensively the topic. 

Did this answer your question?