en DE RU
Request Demo
Bloomreach Acquires Exponea. Learn more

Table of Contents
    Listen this article on:

    How intelligent is your A/B testing?

    Mar 01, 2017 Jozo Kovac 2 min read

    Everyone in marketing loves A/B testing.  It’s a simple, brilliant idea: Everything can be improved if more variants are tested. It is especially loved in ecommerce as marketers test designs, claims, call-to-actions over all available communication channels.

    But then there is the down side. By definition, many A/B testing variants yield sub-optimal results. The longer bad variants are tested, the higher the losses will be.

    Can we reduce the A/B pain?

    Is it somehow possible to make A/B testing easier, more automatic, and minimize losses in the process? Yes, we can and this is where we can get some help from AI, probability theory, and the set of algorithms called Multi-Armed Bandits.  (https://en.wikipedia.org/wiki/Multi-armed_bandit).

    The idea comes from gambling on row of slot machines – Which lever should I pull to get the highest reward? Does it matter what order I play the machines? What’s the final payout going to be?  It’s the same as in A/B testing – Which testing variants should be used to get the highest rewards?

    Do you trust the algorithms?

    An algorithm is only good if it can be tested and verified. We’ve made a series of tests to see if the AI with the multi-armed bandit deserved our trust. Here’s the summary:

    Can AI find it if we make variant A better than B? YES – and it was very, very fast. After a while, it gave a 99% weighting to A. 

    1 (1)
    Can AI react if we switch the coin and make B better? YES – the remaining 1% of the B variant helped AI to identify that B was getting the better numbers. It was subsequently tested more and ultimately was appointed as the winner.

    2 (1)
    Can we trick AI with two identical variants? NO — AI quickly found the balance and showed that both variants scored equally.

    3 (1)
    The results from this test convinced us that we could trust AI to optimize A/B testing, opening up a really wide range of options for Exponea.

    Test drive the new A/B Bandit in the sky

    Following our tests, we have implemented an advanced AI variation of Multi-Armed Bandits algorithm in the Exponea marketing cloud. I believe it will change the way A/B testing is run. As an Exponea user, you simply need to decide what goal to optimize: It can be a purchase, a pure click, or any other event such as registration, subscription to a specific service, and opening emails. There’s no longer a requirement for you to define the precise percentages for all variants. Just create your choice of variants and let the algorithms choose the best ones.

    For more information on putting this new intelligent A/B testing function to work, just talk to your Value Delivery person or a sales rep. 

    Watch Exponea demo video!
    Explore the Customer Data & Experience Platform B2C Leaders Love to Use

    MISSGUIDED Victoria Beckham Desigual
    ebuyer River Island

    We rely on cookies

    to optimize our communication and to enhance your customer experience. By clicking on the Accept and Close button, you agree to the collection of cookies. You can also adjust your preferences by clicking on Manage Preferences. For more information please see our Privacy policy.

    Manage cookies
    Accept & close

    Cookies preferences

    Accept & close
    We use cookies to optimize our communication and to enhance your customer experience. We also share information about how you use our website with our third parties including social plugins and analytics. You consent to our use of cookies if you continue to browse our website. You can opt out of our cookie use on the Do not Sell my Personal Information page. For more information please see our Privacy Policy.