A/B Testing & Optimization FAQ

Read frequent questions and get answers right away.

Lari Lehtonen avatar Romain Sassareu avatar
26 articles in this collection
Written by Lari Lehtonen and Romain Sassareu

Can I schedule tests?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

What do I need to create a test?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Can I test two segments against each other?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Can I only include new users to my tests?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

What happens when I end a test?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Is there a limit to the number of tests I can simultaneously run?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Does A/B Testing negatively affect SEO?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

What Nosto products can I test?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Why is my test not significant?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

When shall I analyze the results of my tests?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

How many variations can I set up in a test?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Can I set up a control group?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

What if my test is not significant?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Why do you show improvements in form of intervals?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Can you run tests in other channels than my online store?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

When do you decide when a test is significant?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Can I optimize multiple sections of my online store at the same time?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Do you use a Frequentist or a Bayesian statistical approach?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Do you exclude outliers?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

What different test types and methodologies do you support?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

How do you continuously optimize?

Lari Lehtonen avatar
Written by Lari Lehtonen
Updated over a week ago

What Optimization Goals can I choose?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

How does the attribution work?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Can I edit campaigns when they are part of a live test?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

Can I optimize for multiple goals?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago

What is a boxplot?

Romain Sassareu avatar
Written by Romain Sassareu
Updated over a week ago