A/B Testing & Optimization FAQ

Read frequent questions and get answers right away.

26 articles
Lari Lehtonen avatar
Written by Lari Lehtonen

Can I schedule tests?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

What do I need to create a test?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Can I test two segments against each other?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Can I only include new users to my tests?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

What happens when I end a test?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Is there a limit to the number of campaign tests I can simultaneously run?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Does A/B Testing negatively affect SEO?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

What Nosto products can I test?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Why is my test not significant?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

When shall I analyze the results of my tests?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

How many variations can I set up in a test?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Can I set up a control group?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

What if my test is not significant?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Why do you show improvements in form of intervals?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Can you run tests in other channels than my online store?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

When do you decide when a test is significant?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Can I optimize multiple sections of my online store at the same time?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Do you use a Frequentist or a Bayesian statistical approach?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Do you exclude outliers?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

What different test types and methodologies do you support?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

How do you continuously optimize?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

What Optimization Goals can I choose?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Can I edit campaigns when they are part of a live test?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

Can I optimize for multiple goals?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

How does the attribution work in A/B testing?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago

What is a boxplot?

Lari Lehtonen avatar
Written by Lari Lehtonen. Updated over a week ago