55 A/B Testing Best Practices Every Marketer Should Know - Acquire Convert


That’s a massive checklist 🙂

Re: 36. Don’t Surprise Your Regulars

Depending on what you are testing, one other idea for more accuracy is to target only first-time customers, who have 0 purchases instead of just new visitors.

As a returning visitor doesn’t necessarily mean a regular or a returning customer.

You can achieve this by setting up a cookie when someone buys and then, target the experiment to those that do not have that cookie set.

That’s an excellent article with great insight. Provides a full depth knowledge on the best practices about A/B testing. It is very important to check out which design is best suit to your website and which makes more effects in driving traffic and conversion. read each point carefully and will try all one by one for my website. Thank you for sharing such a great guide with us and keep doing a good job.

Thank you so much for this list. I will post it in our Seller’s Bootcamp at icraft.ca. Any new business starting out can at least focus their resources elsewhere. Cheers, Christine

A huge list with many, many very good points. Should be a starting point for any aspiring A/B tester!

With that said, you appear to contradict yourself between points #34 and #46. If you did #34, you wouldn’t need to worry about #46. However, the dirty secret few split testing practitioners would admit to, is that #34 is near-impossible to follow due to pressures coming from all sides to report on the results periodically and to jump to action as soon as the results appear to be “good enough” or “too bad to continue”.

This is the so-called “waiting for statistical significance” approach, which is, of course, a recipe for disaster. Using “reaching statistical significance” as a stopping rule makes it so #47 and #48 on your list can’t be followed at all, unless the test was designed as a sequential test (which would contradict your advice in #34).

I’d suggest an update of the article to introducing some sequential testing theory and to finally get rid of the fixed-sample approach, which is good in theory, but which few can follow in practice. It quite inefficient as well – alternative approaches such as AGILE A/B Testing deliver results 20-80% faster, with the same statistical guarantees.

Speak your mind

Your email address will not be published. Required fields are marked *