IT’S ELECTION SEASON

And the Email Mavens have a candidate to endorse. Want to know who we think ought to win? We proudly endorse…

The Champion of our A/B Testing Challenge! 

The last few weeks we’ve run some head to head challenges on our Email Party to test winning variables to increase our click rates. Today, we’ll show you the metrics behind each test, and give you the roadmap to test these same elements for your unique customer base. 

Read until the end and implement what you learn to steal our A/B Testing strategy to make every email you send more effective than the last one you sent.

***

Welcome to the Email Mavens blog—the place for winery owners and marketing pros who want to sell more wine online by sending better marketing emails.

We’re so excited to show you the results of our A/B Testing Challenge, but before we can dive into the metrics we first gotta remind you about the method we use to run A/B tests, for both our emails and for our winery clients.

We use the scientific method to design all our A/B tests.

We start with a hypothesis. We design our tests to prove the hypothesis. And then, we apply what we know so that we can continue testing. 

Are we on the same page here? 

PERFECT.

Now, on to the actual tests that we ran, the results, and what we are doing with what we’ve learned… plus, how you can steal these same tests for your winery NOW to make every email you send better than the last.

The tests we ran were designed to increase CLICK RATE of our emails. To generate more clicks, you test the email creative.

Our A/B Testing:

1. Button Before Body Copy vs. After

The first test we ran was the location of our Call to Action. Erica has long preached that your CTA should be RIGHT after the headline in your email content. But earlier this year, our director of Email Marketing Success pushed back on this. She wanted to know (and rightfully so), if this strategy had data to support it. So, we started being more intentional in A/B testing the strategy for our winery clients AND we started testing it in our own email content. 

Hypothesis: Click rates would be higher with the CTA right after the headline.

The Test: Placing our button right after the headline in version A, and placing it after the descriptive body text in version B.

The Result: WE TIED?! 

While Klaivyo’s report suggested that the version with the CTA after the intro was the ‘winning’ variant it had the exact same number of clicks against the exact same number of opens. The metrics were skewed because of the successful delivery total, increasing the overall ‘rate’ but not the actual click performance. 

What we’ll do with what we know:

This is worth continually testing for our audience, and it’s worth testing for yours as well. 

As an aside, we subsequently have tested this again and had a clear winning variant with the CTA right after the headline. But I would argue that we have some work to do with our CLICK RATE in general to make this become statistically relevant.

Here’s what you can do with this intel:

Test it for yourself.

Design an a/ b test where your button is placed directly under your headline, vs. a version where the button comes AFTER any additional descriptive text. If you consistently find that one outperforms the other, you have a winning formula for your marketing email layouts.

And, if you don’t find that, maybe go back to your copy core and ensure that your subject line, preview text, headline, and call to action give your customer enough intel to actually want to click your button.

2. GIFs vs. a Static Hero Image

If you watched an earlier episode of our YouTube channel you probably already know, the Email Mavens are big fans of using GIFs to create more engaging email content, both for our brand and for our winery clients when we can! But do they truly out-perform static images with our audience? Let’s find out.

Hypothesis: Animated GIFs will generate higher click rates than  static hero images.

The Test: We used an animation from our YouTube content vs. a still image from the content, both with a play icon to show that they’re worth clicking on to get more.

The Result: The animated GIF had a significant lift in click rate (195.79%).  

What to do with what we know:

Well, we wanted to go a little further with our testing. The animation was pretty salient. Would an animation from a video vs. an animation of the artwork FOR the video yield the same results? 

We gotta test it!

GIFs A/B Test #2: Animation from Video VS Animation of Artwork

Hypothesis: The action shot of Erica FROM the video will inspire more engagement with the video content, not just the artwork itself. 

The Test: We used an animation of our videographer JD zooming in on Erica saying ‘Oh Shit, It’s OND!’ vs. the artwork from the episode with an animated PLAY icon.

The Result: WE WERE WRONG! 

But, this wasn’t a perfect A/B test because we didn’t isolate ALL the variables. Perhaps if we had tested the animation from the video with a play icon over it if the results would have been any different. Alas, no time to test that before it was time to write this blog post, but maybe for a future episode! 

What YOU can do with what you know now:

Read our post on how to get more clicks with videos and gifs to help you create some animated ‘video’ effect images. Then test them against their static counterparts to identify if you get a lift in engagement for your winery with a little movement! 

3. HTML vs. Plain Text

The final test was a full creative test. We went with a full html email party leveraging our brand template vs. a plain text letter from Erica. 

Hypothesis: We’ll get more clicks with more engaging creative (aka not a plain text email).

The Test: We went with a full html email party leveraging our brand template vs. a plain text letter from Erica. We kept the copy almost exactly the same, but in one we had an engaging hero image and our brand colors and fonts for headlines, buttons to click to watch instead of plain text embed links.

The Results: While the results weren’t statistically significant, we had higher deliverability with plain text email and a higher overall click of open rate. The flashier HTML email had a slightly higher open rate which surprised us, but fewer total clicks.

What we’ll do with what we know now:

Frankly, this move is less of an A/B test and more of an overall strategy adjustment. Every once in a while, mixing up your creative with a plain text email can help your overall engagement improve and capture subscribers who might otherwise miss your communications. 

What YOU can do with what you know now:

Add a plain text email to your strategy this quarter.

When you analyze your performance, check to see if you’re getting more opens and engagement from a group that previously had overlooked your emails. We found this to be true when digging into the click performance. We saw some names on there we hadn’t seen hanging out with us for a while, from the plain text letter version specifically. You might find this to be true for your own brand.

***

It’s obvious that there is more testing to be done… and the good news is, we are always A/B Testing around here and so can you!

Take the inspiration from today’s learnings and results to run your own A/B testing challenge at your winery. If you need even more prompts, you can grab our A/B Testing Cheat Sheet! This handy little download will show you what to test in what order to increase your open rates all the way down to your conversions.

At Email Mavens, we believe that every winery can sell more wine online by sending better marketing emails, and now you understand why we’ve so fervently endorsed our CHAMP – A/B Testing! The BEST way to make your next email better than the last is to A/B test and apply what you learn, so get your hands on that cheat sheet and start testing all the things! 

Until next week, keep pressing send on your next best email.