Which Banner Ad Message Gets the Most Clicks?

10 May 2018 | 49 Comments

Last Wednesday, I announced a new Stonemaier game, My Little Scythe. To help people discover the game, I bought some banner ads on BoardGameGeek (one of the few places on the internet where I actually pay attention to the banner ads).

I couldn’t decide on the best wording for the ad, though, so I sent three versions of each ad to Chad at BGG for an A/B/C test. I figured I would share the results with you and use them to influence the version of the ad I use for The Rise of Fenris in a few weeks.

Before I reveal the results, feel free to guess: Which of the following ads has the best click-through rates (CTR)?

Here are the results:

  • Today: 0.40% CTR
  • May 2-10: 0.42% CTR
  • Now: 0.37% CTR

So, not a huge difference, but when you’re talking about 600,000 impressions, it’s a difference of about 300 people. I’d much rather have those people click through to learn more about the game! My guess is that putting the exact date range for the pre-order (or Kickstarter, as the case may be) creates a concrete sense of urgency.

The one tweak I might make for future ads is either to remove the hyphen from “pre-order” or add spaces on either side of the date range (“May 2 – 10”)–the double hyphen used in different ways just looks weird.

If you’ve ever run an A/B test on ads, what have you discovered about various messages?

Also read:

If you gain value from the 100 articles Jamey publishes on his blog each year, please consider championing this content!

Leave a Comment

49 Comments on “Which Banner Ad Message Gets the Most Clicks?

  1. Hey Jamey,

    Big fan of your games and first time posting here (finally something that I can contribute to!).

    I have done a number of A/B tests with Google Adwords. I am a Project Manager at a web development firm and used to be an SEO specialist. I found that interestingly enough, if you go completely out of the box with your thinking with one of your versions, that often has the best CTR.

    For example, with the whole trade war nonsense going on between Canada and the US, I thought huh, why not try a made in Canada type approach with my C ad and see what happens. Surprisingly (because Canadians do not generally have the same nationalistic fervor of our dear American friends) the CTR for this approach was 17% more than a more typical standard CTA (Call to Action). It might be interesting to try that approach one day with your adds and see what happens. Instead of “Pre-Order + some reference to time”, do something along the lines of “Pony Up Today!”. Sounds lame since I thought of that in 15 seconds but you get the idea :)

    Best of luck with your endeavors! Thank you for all your hard work designing and publishing your games and giving us novice designers information that helps navigate us through the murky waters of board game design :)

  2. I appreciate the analytical approach, Jamey. In the experience I have with CTR in two other industries, it’s tough to measure distinctions where the channel isn’t the best means of driving conversion, registration or other action. That said, I’m kinda surprised – I’d figure that BGG would be a quick and easy path to purchase via Amazon, MiniatureMarket, CSI or other online retailers. Is it that there’s a strong group holdout who love their FLGS? Did you control for time and days and pages displayed (ie, Stonemaier, or worker placement, or strategy games)? The only other thing I can think of is timing, that there’s a lot of signal and a lot of noise going at the same time with the con season rising and lots of talk on KS right now, so maybe the volume you were hoping to drive just wasn’t there yet.

  3. Wow Jamie, the conversation this post stirred up is great. I am just about to start a run of A/B testing ads and this couldn’t be a better time to consider these factors for me. Thanks!

  4. Interesting data. Perhaps, people are already conditioned to seeing “NOW” and “TODAY” and don’t react with a sense of urgency. However, when you put “May 2-10”, everyone can see there’s a window of opportunity. You can see framed dates in other types of advertising — “On sale now until MM-DD” or “Order your _____ by MM-DD” or “On sale from MMDD — MMDD”.

    1. Yes, I think we are conditioned to the “NOW” and “TODAY” as an advertisement Wording that we automatically ignore. Having a Date starts some progressing like: “What date are we having now?” “OH it started already, only 4 days left”.
      Date seems quite a lot better.

    1. Ah there is. Posting again pushed it through, or my browser is just being silly refreshing. Sorry for dbl post.

  5. CTR is CTR, but I always wonder how much it REAAAALLLLY matters in a short advertising campaign. Here’s why: Personality types.

    I always wonder if certain personality types are more drawn to a certain wording, image, or announcement style.
    What if you stop running the other 2 ads, and you end up with LESS net clicks on the same quantity of impressions, because you didn’t have “the other one” that caught my attention.

    I think on “now” vs “today vs “dates” (and I totally picked the dates as the most compelling), it doesn’t matter so much. But on “Animated” vs “Reflective light movement” vs “still” or “Pre-order now” vs “Catch them all now” vs “Click to find out more”… that’s 3 very diff messages that speak diff to diff people.

    If “Animated + Catch them all now” gets the most CTR, but you got Jamey Stegmaiers who dislike animated ads, then you didn’t get the Jameys. On the other hand I largely ignore still ads, and stare like a child at candy with any animated ads. So the still ones are less effective on the John Wrot!s of the world.

    Maybe the best part of the A/B testing is that you’ve got 2 ads that might catch 2 diff types of folks. (…assuming you get lucky and the right one goes to the right person. ; )

    Am I making any sense with that?
    I wonder what research has been done to that end.

    (Taking the red pill: There are folks who will skip this comment because it’s too long. There are others who will read it because it looks meaty.)

  6. Well done, Jamey, and thanks for sharing! Just a benchmark – Across all ad formats and placements Ad CTR is just 0.05%. Btw, I am totally looking forward to the game.

  7. I really appreciate the A/B/C testing you’re doing, and am continually shocked at how many advertisers will order thousands of dollars worth of advertising, and only send me one banner, despite Google DFP’s ability to optimize delivery. I always try to stress the importance, but most publishers just don’t want to go through the effort (and sometimes expense) of creating multiple banner ads. It’s 100% worth it.

  8. I am wondering what the CTR would have been if you had ramped up the urgency as the campaign progressed through its lifecycle. I also wonder if the promotion might have tried a different urgency tactic in the sub head like Guarantee First Wave”.

    I love that you love experimenting.

  9. I work in advertising and I’m really fascinated by the subtle psychology of copy wording. I try myself to implement it in my work when possible. I guess the “today” and “now” are too urgent of words for people on BGG (e.g. I usually go there when I have a quick rule question). The dates may give a less urgent message, and prompt those that see it to click through and revisit the link later. It’s all just conjecture, but interesting to think about!

  10. Are those click through rates comparable to other banner ads you’ve run on BGG? I’d be curious if you have the data on those other ads to also compare against.

    Those seem very low compared to “Industry Average” CTR, which surprises me as BGG is as targeted an audience as you’re going to get. I’ve run a couple MVP experiments validating the ideas using A/B testing via ads and landing pages. My experience was a 2% click-through rate is average across any industry, which I found to be consistent in my experiments.

    But it may just be a BGG culture thing. I’d imagine the conversion rate on a BGG click through is much higher that other channels. So perhaps that 0.4% is better at the end of the funnel than 2% elsewhere?

    1. Evan: It’s on the lower end of banner ad campaigns we’ve run on BGG. 2% seems extremely high, though. When we ran banner ads on BGG for the Scythe Kickstarter, we were told that the CTRs were some of the highest ever on BGG, and the average for those ads was 0.80%.

      1. 2% is par for the course in conventional online advertising. Not bad, not great.

        I’d imagine a large part of it is how “smart” a targeting system the advertising platform uses. Something like google or facebook – those platforms don’t blast ads to anyone and everyone – that have algorithms to target to exactly the people they think would be interested in an ad. Those people are then more likely to click. So the impressions are lower, but more “valuable,” and the CTR is higher.

        I may be wrong on this as I haven’t advertised with them, but I doubt BGG has that kind of targeting intelligence built in. But perhaps if those ads were targeted to only people who have engaged with Stonemaier properties in the past in forums, badges, wants, ratings, geeklists, etc. instead of all 600,000 people using BGG or whatever it is, the CTR would be close to 2%.

        1. I’ve been the advertising manager at BGG for about 12 years. When I started, we had less than a million users per month on the site, and nowadays, we have 4.2 million, but the average clickthrough rate on banners has remained at a pretty consistent .34% over that entire span.

          According to Google’s data on display ads, “across all industries, the average CTR for a search ad is 1.91%, and 0.35% for a display ad”. It seems like the folks referencing a 2% CTR are referencing a different ad type, or lumping all ad types together, as that would be really crazy high for display banner advertising. We do achieve an average of over 1% on our home page leaderboard position, but that’s as uber-targeted to the core audience of BGG as you can get, and ads only run in that spot in very short bursts to avoid oversaturation.

          However, CTR alone isn’t how you gauge whether an ad is effective for you or not, as of course the cost of those impressions weighs heavily. This is what helps BGG ads to be particularly effective for many advertisers, as we’re charging $1 per 1,000 impressions, which is very low compared to other highly targeted interest-based sites.

  11. I’m a big fan of A/B testing and think it’s a great tool. I wish BGG ads had stayed cost effective for us!

    We found that bright colors, simple verbiage, and a short call to action was the best way to get interactions. ‘Learn more’ or ‘Register Now!’ was significantly more effective for us than other options. The more info that we gave in the ad, the less likely we were to get a click-through.

    However, when it came to converting clicks to registrations, the more informative ads were better. So, be vague but interesting, and you get more people to click through, but be specific and informative, and the ones who do click through are more likely to convert.

  12. There is also error clicks you need to take in to account. Margin if error. So those results would not be considered scientifically significant. You need a bigger difference. The banners visually are almost identical, next time maybe try 2 different graphic designs. And you could put 2 different messages on each. Then you might be a bigger difference.

    1. You’re right, it’s good to test different images. But I wanted to reduce the variables so I could pinpoint which message is best. In the future I’ll use the date message and test different images.

      1. Yes, personally the date-banner seemed to instill a sense of urgency, a genuine one too. People are bombarded with the marketing terms “Today” and “Now”. I would not be surprised if *wise* people are skeptical about those terms, as most of the time it is not genuine (unlike your pre-order window). Well, I guess that depends on the target audience.

  13. Do you have any theories about why the one did better?

    At first I thought the “order now!” One would win, because it had the strongest call to action wording.

    But in retrospect, I wonder if the date one induced stronger FOMO with its explicit end date.

    1. Dan: My theory is that putting the date in the banner created a concrete sense of urgency, whereas with the others, people could easily assume the pre-order will be happening for an extended period of time.

      1. Makes sense to me. When I see a “Now” or “Today” unless it suggests expiring at some specific time now/today, I assume it is a come on and I’ll see it again tomorrow, etc. The dates tell me something I should remember if I care about the offer.

        1. I would also have thought that “Now” would have the better CTR. However, seeing these results, I think that the dates help it not seem like empty hyperbole. As a consumer, I could be seeing “Today” or “Now” in any ad at any time and it doesn’t mean much. It will always be “Today” or “Now” or “Act Fast” in an ad. It may seem urgent to the ad maker, but to the to the consumer “Today” or “Now” is an empty threat. Whereas “May 2-10” says, “You can trust me. I am being transparent. This is a limited thing and it’s during these dates which you are in range of at the moment you see the ad. If you click through, you are guaranteed to be able to take action.”

        2. my response is the same Scott, unless I know a kickstarter or something really does end today, I assume it’s a plot. The real dates give me the info to remember and weigh in terms of my buying decision. Jamey, I don’t think the dash in both lines is distracting, it’s not like you are using them incorrectly… :)

  14. In response to the above comment, I rarely click on banners although I do notice them on BGG. On Facebook I blanket ignore all advertising, so for me banner ads are a better investment that Facebook.

  15. I would be interested to know the timing of the clicks. In my mind, “Pre-Order Now!” makes it seem more urgent, like I might miss it if I don’t do it now. As a customer, I like “Pre-Order May 2-10” because I can mull it over for a bit. So, I wonder how many people waited until May 10. Was there a spike in the click-through rate toward the end?

  16. Not be a debbie downer. But as a marketing major, I just don’t believe in banner ads. Do they get results? Some. But very little. Do they increase exposure? Not really, banner blindness is strong and impressions is a shallow metric. Are they worth the money? Very very very rarely is there a case in which money would have a better ROI in banner ads than a simple well targeted Facebook ad.

    I understand there are those who disagree with me and that is ok. And your business is your own and I am just a man typing on his phone because his cat wont get off his key board. So it is entirely possible that I am wrong.

    1. That’s fair, though this isn’t actually a discussion about whether banner ads are good or bad. It’s a discussion about if you have a banner ad, which message on the ad is the most effective.

      1. Understood, just throwing in my two cents on a connected topic.

        More towards the subject as hand, I do think your experiments are well founded and generally a good business practice.

        Perhaps an alternative option is to conduct the same or similar experiments on Facebook, where you can get a week of metrics for $5 for each image simultaneously and use the resulting insights to influence your banner ads

        1. Not to continue the off-topic discussion too far, but: Before dismissing banner ads outright, you still have to consider the placement. It’s a visible spot where you can target your audience really, really well (Board games to board gamers on BGG). I’d expect it to be especially effective because of how much BGG appears to curate that ad’s location to keep it all relevant and rotating, and not typical banner ad junk.

          Anyways, Jamey wasn’t testing that. :-)

          1. I am inclined to agree that the banner ads might be unusually effective on BGG. For instance, even when I’m supporting BGG and *could* remove the ads, I intentionally do not do so because I WANT to see them on that particular site. :)

          2. Include me in the “relevant banner ads on a relevant site can work” crowd. I know the evidence is anecdotal, but like educatingryan, I tend to leave my ads enabled on BGG because I -want- to see them. And while I have to admit I don’t purchase things from every ad, I -do- tend to click on them at least.

            That said, I’d discourage people with well established brands/products (e.g. CoolStuffInc, Terraforming Mars, Z-Man Games, etc.) from using the company. The best use seems to be generating quick hype for a product release/KS project (e.g. My Little Sythe’s preorder window). Maybe once or twice a year, CSI or somebody -might- want to let people know “hey, we exist”, but that’s a big might. Still, BGG would be the place to do it.

      2. I would disagree with Zach on this one but only because we are talking about Board Game Geek. I donate enough to them to turn off the adds on their site and I don’t. I actually use Board Game Geek adds to let me know new things that are coming out that I might have missed. I can’t be the only one who does this.

      1. Orion: I haven’t fully tested it, though I did boost the announcement of My Little Scythe on Facebook for $50 and it got a ton of likes, but I can’t find the CTR data for it. And it was boosted only to people who already liked the Stonemaier Facebook page, while the BGG ads reached people who don’t directly follow us.

        1. Would be neat to test a FB ad targeting people who like the BGG FB page but excluding those who already like Stonemaier. I guess it wouldn’t be similar enough to A/B test though (since there are people on the BGG website who already follow you.) Although it could give a rough insight into FB ads vs. website banners.

© 2020 Stonemaier Games