# A/B Testing **Significance Calculator**

Would you like to know whether your test results are also significant and therefore really meaningful? With our significance calculator, you can see at a glance which result is significant and which uplift or downlift could have occurred by chance.

**A / B Test Significance Test**

**Confidence** is the probability that the measured difference to the original variant (control) did not arise by chance, but rather due to the test arrangement.

**Significance** is the opposite, i.e. the probability that the two variants have no structural differences and that the measured differences are purely coincidental.

The **confidence** is thus the inverse probability to the significance. Significance and confidence together always result in 100%. For example, with a calculated significance of 20%, the confidence is 80%.

**Significance** considerations are a good tool for evaluating your test results. With our significance calculator you can find out the probability with which the results are meaningful or were measured purely by chance.

In general, the following applies: The smaller the determined significance, the less likely it is that the measured uplift or downlift occurred purely by chance. Conversely, a high confidence means that the probability of a random result is rather low.

## A/B Testing Significance Calculator

##### Auszug aus unseren Referenzen

## Digital Loop-Workflow

#### 1. Collect Ideas & Hypotheses

#### 2. Prioritize Ideas & Hypotheses

#### 3. Implement and Execute Campaigns

#### 4. Evaluation & Analysis

## Our Services in A/B Testing & Personalization

#### 1. Target Definition

– Selecting the tools that fit your needs best (e.g. Google Optimize, Optimizely)

#### 2. Side Variations

#### 3. Execution of the A/B test & analysis of the results

– Evaluation of the results for the derivation of new measures for your website

## Our A/B Testing & Personalization Team

#### Jhonatan Arcos

- Full Stack Entwickler

#### Wan-Yu Lee

- 6 years of experience in data analytics & market research
- Adobe Analytics certified expert

#### John Munoz

- 10+ Jahre experience in Digital Analytics, MarTech & Tech SEO
- Google Analytics & Adobe Analytics expert

#### Vladimir Stashevskiy

- 6 Jahre expericience in Digital Analytics, MarTech & Digital Marketing
- Google Analytics expert

## Discover more about A/B Testing & Personalization

#### A/B Test Runtime Calculator

A Test should not take too long if possible – 1-2 weeks are ideal.

#### Blog Articel

“Your key to success: A/B Testing & Personalization”

#### Google Presentation

Templates on the topic: “Conversion Rate Optimization with the right A/B Testing workflow”

## Further questions about A/B Testing & Personalization

##### Which elements are tested during A/B testing?

##### Is A/B testing only performed on websites?

In addition to websites, A/B testing can also be performed for emails, PPC ads, and CTA buttons.

### What is a null hypothesis?

**examine hypotheses.**There are

*two typical concepts in*hypothesis testing: the

**null hypothesis**and the

**alternative hypothesis.**Usually, the null hypothesis indicates that the performance of the two variants A and B are identical, while the alternative hypothesis states that they are not.

### How often should I run A/B tests?

*we recommend continuous testing.*You should have a

**clear goal**and

**enough page visitors,**in order to achieve

*statistical relevance*within an acceptable period of time.

### Client vs. Server Side Testing?

**A/B testing.**This aspect is often overlooked. Nevertheless, it should be chosen

*based on your needs.*

**Client-side:**commonly used to*optimize conversion rates*in marketing or funnel, for example by creating page variations directly on the users’ browser.**Server-side:**when you need to test more in depth in relation to the visual changes, such as*products (features)*or experience for engagement, retention and more.

### Do A/B tests have negative effects on SEO?

*negative impact on SEO.*The truth is, that

**websites rather improve through A/B tests**which results in better ranking.

### How many users do I need for trust-worthy testing?

*wrong interpretation*of statistical significance is one of the

**most frequent and serious mistakes**committed in A/B testing. Usually the minimum required traffic is calculated using the following key figures:

- The conversion rate of our control variation (variation A)
- Minimum difference between the conversion values of variations
- Confidence level
- Statistical “Power”

For a sample calculation please use our runtime calculator on this page.

## Interested in our service?

## Contact us!

Steinsdorfstraße 2

80538 München

089 – 41 61 47 83 0

089 – 41 61 47 83 4

info@digital-loop.com