It’s great how Bronto will automatically compare A/B splits and pick the winner for you, but sometimes you just want to try something out. You may have a hunch you want to try out – Would people react better to the color blue? You may feel the need to continually check certain variables – What’s a good length for the subject line? Or maybe you’ve hit a wall with the status quo – Ah ha, this is the perfect case for champion-challenger.

A fight against the status quo, especially when it’s not something everyone is on-board with, is definitely champion-challenger territory. Let me explain why. A champion-challenger tests involves running a new challenger group against the established champion and nothing more. That means that we are only going to try a change on a small percentage of your list. Hence, there is no risk of a large number of people getting this new variant.

If the test goes well for the challenger, you can try another test with a larger audience or move on to an a/b winner split. In the mean time however, we are limiting exposure and just getting a rough idea of how your audience will react with a champion-challenger split test.

Let’s look at another example. What if you have to change your email format to include more ads. Rather than roll out what just seems like it might be the best version of the new format, you can run tests for a few weeks with a challenger using the new ad-heavy format. This will help you to refine your strategy in order to provide an optimal recipient experience when you roll out the change to a larger audience.

So try out champion-challenger split tests. You may find that a quick test on a small number of contacts is the best way to ease into something new.

Adam Covati
Product Manager at Bronto

Related posts:

1. Can’t Decide On A Subject Line? Split ‘Em! Do you ever have a problem deciding between a...
2. A Look At Contact Status Within Bronto, your contacts can be labeled with a number...
3. Getting Comfortable With New Webforms Bronto’s Spring 2010 Release is finally here, and with it,...

Ever have an argument with a co-worker about which subject line would be better? I know we have them here at Bronto. Heck, we’ve even had company-wide contests to drum up fresh ideas for subject lines – anything that might take it to the next level.

When you have that many ideas about how to hook people into reading your message it can be hard to know which to use. Even when everyone agrees on which to use, it’s still just a guess. This has been brought to light in a recent post by DJ about first name personalization. Some of our clients have taken DJ to task and are doing the tests themselves (see the comments on the above linked post).

The new A/B split feature that we introduced in the Bronto Fall 2008 Release has come to the rescue. This is just what A/B splits are all about, testing and learning something new. One of the best part about A/B splits is that they are so easy to do.

Almost any time you send a message you should be doing an A/B test. If you always assume that you are doing what’s best for your email marketing program then you aren’t getting any better. Trust me, there’s always room to improve. In Bronto, we make it very easy to send an A/B Split. It is right there (look!) waiting for you to click it.

A/B splits don’t just come in one flavor. You can run A/B/Winner or Champion/Challenger tests.

A/B/Winner

A/B/Winner tests send out emails to two or more test groups with the best performing option being sent out the remainder of your list at some designated point in the future.

Champion/Challenger

Champion/Challenger tests just split up all of your list amongst two groups. The Champion/Challenger tests are great for trying something new with a small percentage of your group.

At Bronto we make this quick and easy, with powerful reports that allow you to see where each version excelled. But if you don’t have that built into your email marketing tool, that doesn’t mean you can’t run A/B splits. Just do two different deliveries and print out the results for each.

A/B splits can be really fun. Place a friendly bet, or maybe just your pride, on your subject line. But don’t bet too much, you may be shocked by the results. Just ask DJ!

Adam Covati
Product Manager at Bronto

Related posts:

1. Challenging the Status Quo: Champion-Challenger A/B Splits We’ve talked about A/B splits a number of times –...
2. Can’t Decide On A Subject Line? Split ‘Em! Do you ever have a problem deciding between a...