It’s great how Bronto will automatically compare A/B splits and pick the winner for you, but sometimes you just want to try something out. You may have a hunch you want to try out – Would people react better to the color blue? You may feel the need to continually check certain variables – What’s a good length for the subject line? Or maybe you’ve hit a wall with the status quo – Ah ha, this is the perfect case for champion-challenger.

A fight against the status quo, especially when it’s not something everyone is on-board with, is definitely champion-challenger territory. Let me explain why. A champion-challenger tests involves running a new challenger group against the established champion and nothing more. That means that we are only going to try a change on a small percentage of your list. Hence, there is no risk of a large number of people getting this new variant.

If the test goes well for the challenger, you can try another test with a larger audience or move on to an a/b winner split. In the mean time however, we are limiting exposure and just getting a rough idea of how your audience will react with a champion-challenger split test.

Let’s look at another example. What if you have to change your email format to include more ads. Rather than roll out what just seems like it might be the best version of the new format, you can run tests for a few weeks with a challenger using the new ad-heavy format. This will help you to refine your strategy in order to provide an optimal recipient experience when you roll out the change to a larger audience.

So try out champion-challenger split tests. You may find that a quick test on a small number of contacts is the best way to ease into something new.

Adam Covati
Product Manager at Bronto

Related posts:

1. Can’t Decide On A Subject Line? Split ‘Em! Do you ever have a problem deciding between a...
2. A Look At Contact Status Within Bronto, your contacts can be labeled with a number...
3. Getting Comfortable With New Webforms Bronto’s Spring 2010 Release is finally here, and with it,...