Cloud

A/B testing: Math men are the new mad men

NOTE: GrowthBeat -- VentureBeat's provocative new marketing-tech event -- is a week away! We've gathered the best and brightest to explore the data, apps, and science of successful marketing. Get the full scoop here, and grab your tickets while they last.

This is a guest post by CEO and co-founder of Optimizely, Dan Siroker.

The role of today’s marketer is changing fast. A/B and multivariate testing that years ago were made possible only by developers and technical teams are now readily accessible and used widely by entire marketing teams. A recent generation of cloud technologies, which includes a suite of Saas-based A/B testing tools, allows a level of speed and sophistication of real-time feedback loops that is unprecedented in the marketing world.

What does this mean for marketers? Well, the traditional notion of a good marketer is long gone. What previously held true for the profile of a strong marketer included clever ad copy creation and a tasteful sense of design. Now, it means the ability to quickly handle a deluge of data, and leverage a wide spectrum of measurement tools that have made available to them. Marketers are not only measured by their strong and savvy intuition but by their nimbleness in using data and empirical analysis to quickly drive top line decisions in their organization.


Editor’s note: The author, Dan Siroker, of Optimizely will be speaking about A/B testing at our CloudBeat 2013 event next week, which is about how large businesses are embracing the cloud. Joining Siroker on stage is Elizabeth Allen, a customer who implemented the Optimizely A/B technology while she worked at gaming publisher IGN. She’ll talk about the challenge she had as an entry-level employee to get buy-in from her superiors. She did it by kicking off an internal competition within IGN, where other employees were asked to appraoch her with various hypotheses they wanted to test with A/B. Successful results were then publicized via the internal collaboration tool, Yammer. Register for CloudBeat here!


So what has caused this? Have marketers become bored with the traditional marketing process? Is there no further need for intuitive content and design? Have we gotten to a point where we’ve reached the pinnacle of marketing tactics? Probably not, but here are some key causes to consider.

Tools today enable us to leverage what’s called a feedback loop, a profoundly effective tool for changing behavior. One feedback loop, A/B testing, enables marketers to directly measure the impact of their work. Marketers become better marketers by using data to refine their art. This makes the marketer better and it makes their business better because the decisions they make are no longer based on intuition but based on facts.

The true power of feedback loops is not to control people but to give them control. The ideal feedback loop gives us an emotional connection to a rational goal.

What role does creativity play in a world ruled by data?

A/B testing is only as good as the hypothesis you test. As a marketer your role has shifted from focusing on prescribing the right answers to asking the right questions. Most businesses today think their websites are pretty good. In reality, they are not. The average conversion rate across the web is 2 percent. That means 98 percent of visitors to your website will not convert into customers. The solution to this problem is admitting that often times a dramatically new experience will work much better than your existing one. Fortunately A/B testing is a great way to test these new experiences and through the feedback loop marketers are now given the freedom to be even more creative since you know exactly the impact of the changes they are making.

The consumerization of enterprise

The scalable nature of the cloud implies a lower infrastructure cost on the vendor side; savings which are almost immediately passed on to the end user. This shift in the way technology is purchased, serves two purposes:

  1. It is they key to empowering the transformation of the traditional marketer to the data-driven marketer.
  2. It provides the feedback loop that rationalizes the transformation.

Given this new context, good technology must deliver quick and seamless iterations with zero downtime or technical nuisance to its user. New services must integrate seamlessly with the marketing cloud; the ecosystem of tools that marketing teams rely on every day. A vendor’s ability to make iterations to their product quickly holds a direct correlation to a marketing teams’ ability reach goals by discovering value on an ongoing and daily basis. The ability to do this is a clear mark of a vendor’s success.

It doesn’t matter if you are a small business or a large enterprise. Measuring and using data helps you and your company grow. Today, there is a “perfect storm” of marketing technologies leveraging the cloud to enable businesses to use data to make decisions that in the past were made based on the HiPPO– the Highest’s Paid Person’s Opinion.

Whether the data-driven testing spirit of a marketing team is driving the delivery of personalized experiences or an overall increase in revenue; all may be futile without an awareness of the right strategy for your organization. This requires a complex balance of organizational buy-in, cultural and psychological shifts, a sustained, collaborative, and innovative approach in seeking the right optimization strategy.

Dan Siroker is the CEO and co-founder of Optimizely. The original inspiration for Optimizely came from his experience as the Director of Analytics on the Obama 2008 campaign. Prior to Optimizely, Dan served as Product Manager for Google Chrome and AdWords. He has been featured on Bloomberg, the Wall Street Journal, New York Times and Wired and recognized by Forbes as Top 30 under 30 in Technology in 2013. His book on A/B testing A/B Testing: The Most Powerful Way to Turn Clicks Into Customers is available on Amazon today.

0 comments