en DE RU
Request Demo
Bloomreach Acquires Exponea. Learn more

What are contextual bandits? Is A/B testing dead?

Jan 23, 2020 Jordan Torpy 5 min read

E-commerce shops live or die by the customer experience. If they can optimize their sites for their customers, they can thrive. If not, they’ll have a short future ahead of them. 

To provide the best site to your customers, you need to understand them. You need to know what’s important to them. Do they like hunting for sales? Or do they want sustainable materials? Maybe they prefer exclusive or premium products?

These preferences typically depend on the individual customer. 70% of your customers might be bargain-hunters, 30% want eco-friendly products, and perhaps 0% want luxury items. If you could show the right message to the right customer, you could significantly improve their shopping experience.

Exponea’s new solution, contextual multi-armed bandits, can pick the right customer for a given message. This can radically change the way e-commerce stores test their messaging and optimize their sites. 

What’s a Multi-Armed Bandit Problem?

Exponea’s contextual bandit personalization is a solution to a problem: the multi-armed bandit problem. So what exactly is the problem? 

Bandits, in this context, refers to the kind of slot machines you’d find at a Las Vegas casino. The problem is a hypothetical one: imagine you have a limited amount of money, and a large amount of slot machines to use that money on. 

Some slot machines would probably pay out a little more, and some a little less. You’ve got to find out which is which as fast as possible, because you’ve got a limited amount of money. You need to divide your effort between finding which machine pays out the most, and milking the highest paying machine you’ve found so far.

This is (a simplified version of) a multi-armed bandit problem.

You can map this same problem onto e-commerce. Your limited amount of money becomes your customers (since there’s a finite number of them). Your slot machines become your messages or your offers (since some will work better than others for specific customers). The problem is determining which messages to show to which customers (where to spend your money).

There have been a number of proposed solutions to the multi-armed bandit problem over the years. Exponea’s solution is tailored specifically to the e-commerce variant, and it introduces context.

Introducing Some Context

Let’s go back to our original example of slot machines. When we first introduced the problem, we were only considering the payout of each machine, and the initial amount of money we had. 

Now imagine that each machine responds differently to each person sitting in front of it. And the weather makes an impact as well. So does the drink of the person at the machine. The context becomes important.

So now you don’t need to just find the best overall strategy. You need to find the best strategy for the given context. 

It’s the same for e-commerce. Instead of trying to find the best overall customer for a particular message, you want to try to find the best customer for a particular message, while taking context into account.

Exponea’s experience and expertise in e-commerce mean we have unparalleled domain knowledge in terms of what context is important. This allowed us to develop our solution to the problem, which we call Contextual Bandit Personalization.

[VIDEO] What are Contextual Bandits?

Our Product Manager and resident AI expert, Robert Lacok, explains Contextual Bandits in this video. 

Contextual bandits are like the Sorting Hat from Harry Potter - you put it on the head of the customer, and it tells you which variant is right.

Context includes historical data about each customer, such as clicks on the website, past purchases, and opened emails, as well as data from their ongoing session, such as what the customer just searched for. It aggregates this information and selects the correct variant based on this context. 

Then something happens – the customer either converts or they don’t. It can be a click on the banner, a newsletter subscription or a purchase later on – basically any other action we’re optimizing for. The contextual bandit – the sorting hat – takes this as feedback, and learns which kinds of contexts paired with which variants lead to conversions, and which don’t. Over time, it learns to make much better decisions.

Is A/B Testing Dead?

Currently, the common practice is to perform an AB test and ask the question – which variant of messaging works the best for the entire customer base. Working with the previous example, where 70% of customers are interested in sales, and 30% in organic materials, we might find out that the winning variant would be a banner about sales. Once we finish testing, we might also decide to use that variant for the entire customer base. 

However, by doing that you just misunderstood 30% of your customers, and showed them something they don’t care about. Moreover, you missed an opportunity to direct them towards products from sustainable materials, which might have a substantially higher profit margin. 

Solving this problem could be as simple as finding a segment of customers who bought such products in the past, or purchased from brands who make sustainable goods. 

Contextual Bandits solve problems like this automatically. They reframe the AB test question (What variant works the best for everyone), and ask instead: What segment should I show this variant to? 

This is fundamentally different, because instead of wondering whether this use case works or doesn’t work for the entire customer base, we switch into thinking that every change we want to do is meaningful for a portion of customers, we just need to find the right group (and yes, in some cases the group will be empty when the variant isn’t good). 

Does this replace A/B testing? In some cases it does not, when maintaining multiple variants over a long period of time is not practical, for example running both a 3-step and a 5-step checkout process. However, for many other cases, it is a true paradigm shift in the industry. 

Contextual Bandit Personalization: Exponea’s New Feature

Clients of Exponea’s Customer Data and Experience Platform will now have the possibility to use Contextual Bandit Personalization on their own sites. What does that mean? Let’s look at an example.

Imagine that you did a funnel analysis and discovered that you’re losing customers between the “add to cart” and “checkout” stages. For some reason, not enough customers are completing their purchases.

You want to test out some new messaging, hoping that it will get more customers to convert.

You thought of four variants: one that promotes a safe payment gateway, one that advertises high average customer reviews, one that states your commitment to renewable materials, and one that shows your accreditation by the Better Business Bureau. 

Normally, you would test these by splitting your traffic evenly to each of these variants (plus a control group that saw no variant). You’d have to wait until you had a sufficient number of customers go through your experiment, and then you would display the best performing variant to every customer.

With Contextual Bandit Personalization, Exponea will automatically choose which variant to show to which customer, based on contextual data. 

This means no waiting for sufficient data, and no abandoning variants which could be good for some customers. Which also means a better, more personalized experience for each of your customers.

If you’d like to know more about Contextual Bandit Personalization, schedule a demo with us. We’d love to show you how Exponea can help grow your business.

meet the authors
Robert Lacok
Product Manager
Robert Lacok is a Product Manager for the AI-powered parts of Exponea. His strong background in machine learning, his business thinking, and his drive for innovative solutions help our clients create long-term win-win relationships with their customers using Exponea.
Jordan Torpy
Technical CDXP Specialist
Jordan works closely with the content team and Exponea experts to create material that brings value to readers. With a background in teaching, training, and marketing, Jordan uses case studies, presentations, newsletters and more to illustrate what's possible in the martech world today.

Frequently Asked Questions

What is reinforcement learning?

Imagine training a dog. When he does something good, you give him a treat. When he does something bad, you scold him. His good behavior is reinforced, and his bad behavior is discouraged.

Reinforcement learning is similar, but in the context of machine learning. In very simplified terms, a computer is given a task, but not told how to complete it. It needs to perform exploration to determine possible solutions, trying to maximize its rewards and minimize its punishments.

What are contextual bandits?

Contextual bandits are a type of solution to multi-armed bandit problems. They attempt to find the right allocation of resources for a given problem, while taking context into consideration. 

In our context, that means trying to find the right messaging for a given customer, based on what we know about that customer.

What is A/B testing?

AB testing is a type of experimentation used to compare two variants against each other. In e-commerce, this usually means diverting equal, randomized amounts of traffic to each variant for a determined amount of time. After that time has elapsed, you can compare which variant performed better. 

What is the difference between contextual bandits & AB testing?

AB testing attempts to find a winning variant between two options. Contextual bandits does not attempt to find a winning variant; instead, it tries to find the right variant for a given customer.

What is contextual bandit personalization?

Contextual bandit personalization is Exponea’s way for e-commerce shops to personalize their site for each user, without needing to rely on AB testing. Instead of using AB testing for evaluation, Exponea users can use Contextual bandits. They choose the right messaging for an individual customer based on the information known about that customer, including current browsing data, historical purchase data, and more.

Watch Exponea demo video!
Explore the Customer Data & Experience Platform B2C Leaders Love to Use

MISSGUIDED Victoria Beckham Desigual
ebuyer River Island

We rely on cookies

to optimize our communication and to enhance your customer experience. By clicking on the Accept and Close button, you agree to the collection of cookies. You can also adjust your preferences by clicking on Manage Preferences. For more information please see our Privacy policy.

Manage cookies
Accept & close

Cookies preferences

Accept & close
We use cookies to optimize our communication and to enhance your customer experience. We also share information about how you use our website with our third parties including social plugins and analytics. You consent to our use of cookies if you continue to browse our website. You can opt out of our cookie use on the Do not Sell my Personal Information page. For more information please see our Privacy Policy.