Updated 7 a.m. 6/3/2014 with clarifications and a statement from Optimizely.
Optimizely, a popular service that helps web site owners conduct tests to improve usability, is leaking information about those tests.
“It’s really useful if you’re looking at competitors’ stuff, and seeing what they’re running,” McLaughlin told VentureBeat.
McLaughlin built a site, whatyatesting.com, to show off the vulnerability. For instance, you can see which Optimizely tests Starbucks is conducting, or which tests Healthcare.gov has done. Other sites McLaughlin has scoped out include payroll-processor ADP, freelance marketplace oDesk, domain registrar GoDaddy, and news site CNN.
Optimizely simplifies the process of doing A/B tests, in which a site randomly delivers one of two variations to each visitor, then collects data about which variation visitors click on more. A/B tests can be handy for deciding which color to make a “buy” button, how large of a font to use, what header image to use, and so on.
But A/B tests can also be used to try out new products on a subset of a site’s audience — or to try out new pricing schemes. If those tests reached a wider public — or a site’s competitors — the leaks could be potentially damaging.
Code education company General Assembly, for instance, appears to be testing a price change from $29 to $49 for some of its online classes. And Alexa.com is testing new products, but only with a subset of the worldwide audience, McLaughlin said.
In a response, Optimizely stated:
We recognize that some customers may prefer that their experiment and variation names not be visible in source code, even if it makes integrations with third-party tools a little bit harder. To address this, we will soon release an option for customers to mask Optimizely experiment and variation names in source code.
McLaughlin said he’d brought this vulnerability to Optimizely’s attention before but had received no response. Eventually that prompted him to bring the story to the public, via VentureBeat.
Optimizely said that it had no record of McLaughlin contacting the company.
VentureBeatVentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:
- up-to-date information on the subjects of interest to you
- our newsletters
- gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
- networking features, and more