The number one online travel marketplace, Priceline.com, has evolved its thinking about A/B testing to the point that the practice is more a state of mind than a periodic function.
“We have a tendency to look at A/B testing as function,” Priceline.com’s chief product officer John Caine said at GrowthBeat Monday. “A/B is actually a discipline; everything is an A/B test.”
A/B testing refers to an Internet company testing different variants of a piece of web or app content to discover which is most effective or likely to get clicked.
Things get built at Priceline.com by small, autonomous groups that are not guided by HiPPO (the Highest Paid Person’s Opinion), Caine pointed out. These workgroups at Priceline.com perform thousands and thousands of A/B tests each year to find out which messages work with which user types on which devices.
That’s not to say that Priceline.com doesn’t get some help in their testing from dedicated outside sources. How much testing to do in-house, Caine said, is a big question.
“Where can we selectively stop building and start buying?” he asked. “Today there’s less competitive advantage to building first.”
Caine said his company uses third-party optimization software (Optimizely, for one) to support the build of its personalization system.
Personalization is key if Priceline.com is to hold onto its position as the top travel site.
“The future of travel is being invisible,” Caine said. He believes that as personalization and recommendation engines get better and better (in part, through A/B testing), the number of touches of the user by the tech company will go way down. They simply won’t be necessary.
“Things that were hard will become easy,” Caine said. “You could open up your phone and see a message that says “We’ve got the perfect place for you to stay.”