When it comes to controlled experiments in a marketing context, you really can’t get any simpler than an A/B test. In this kind of experiment, you have a variable that doesn’t change (or your control variable) and a variable that is altered (the treatment variable) with users being randomly assigned to one or the other. So say you have a checkout page on a website that you hypothesize could be altered in a certain way to generate increased sales. You could have your original page (the control) be presented to an equal but random assignment of people while you have your hypothesized “improved” page presented to another randomly assigned group of people. Results will indicate which page performed better at generating macro conversions, aka purchases. No more using obscure blog articles of “optimal website design” or the suggestions of HiPPO’s (Highest Paid Person’s Opinion) who have no data to back up their claims to generate leads; you don’t have to rely on guesswork and the opinions of “experts” when you can test it yourself with your customers. And this isn’t just a one time thing…you should be constantly optimizing your content to appeal to your visitors in the best way possible. Optimal customer appeal = increased revenues!
Another awesome thing about A/B testing is the ability to imply causation. For anyone with a psychology or statistics background, I’m sure you’ve heard this phrase 100 times: “Correlation does not imply causation!” This has been pounded into my brain so many times I swear I dream about it, but it’s for a good reason. You know all of those articles you read that make crazy claims like “How your iPhone causes brain damage” (note: while I made that up on the fly, I have undoubtably come across crazier claims than this!)? Well, usually their claims are based off of correlated events (i.e. because this person used an iPhone and then got brain damage, it must be attributed to the iPhone) when in reality, many other confounding variables could have come into play to cause the brain damage (i.e. they got an awful concussion). With A/B testing, because you’re only altering one thing and because you’re using random assignment (which diminishes confounding variables because of the equal distribution across all participants) you can attribute changes in the participants’ behavior, like an increased likelihood to purchase, to the effect of your changed website element. But let’s take a step back and look at how all of this could be applied to a real-life, non hypothetical example, because some of you might be like…
Unless you’ve been living under a rock, you’re probably familiar with at least some of the content used in the Obama campaign. One of the interesting tactics employed was the use of rather strange campaign emails, with titles like “Hey” or “Wow”. Turns out these unique fundraising emails were actually the product of continual A/B testing. And most importantly, all of the testing paid off, because more money was raised during the Obama campaign than any other presidential campaign in history. Some of the findings seemed impossible; people liked the use of mild profanity, huge lettering and ugly highlighting…without A/B testing, no one would have EVER thought to use such ugly visuals. It’s incredible to me that this wasn’t a normal thing for all candidates to use, especially in the age of technological advancements and social media. I hypothesize that in future elections, candidates will focus a lot more on data driven approaches as well as online presence, especially because places like Facebook aren’t just where young people hang out anymore; it’s becoming a lot more popular with the older generation. Like this article says, it’s all about data, stupid!
Thanks for the read!
*Note: Click here for a funny look at what people really think of HiPPO’s