The New Stack Podcast

Optimizely's Claire Vo Talks Successful A/B Testing at Scale

Episode Summary

When building front-end software, it can be tricky to figure out just what works. As with any page layout endeavor, from the Web to the supermarket checkout line tabloids, there are plenty of nooks and crannies to explore with headlines, graphics, and colors. Any software shop earning money on the Web likely already knows about "A/B" testing: the practice of subtle changing your page design and gathering metrics on its effectiveness at converting visitors versus the existing version of the site. Now that such testing regimes are commonplace in the enterprise, it is inevitable that every team eventually encounters the egregeous and exhausting existentialist crisis that is test management. Gathering metrics for a single test is one thing, but what happens when the entire enterprise is pushing tests across thousands of sites all the time? Claire Vo is Silicon Valley success story: she sold her startup Experiment Engine to Optimizely in 2017. Her particular winning formula was to help solve this exact problem for enterprises: Managing experiments at scale across thousands of sites, and measuring results in order to effect actionable changes overall.

Episode Notes

When building front-end software, it can be tricky to figure out just what works. As with any page layout endeavor, from the Web to the supermarket checkout line tabloids, there are plenty of nooks and crannies to explore with headlines, graphics, and colors. Any software shop earning money on the Web likely already knows about "A/B" testing: the practice of subtle changing your page design and gathering metrics on its effectiveness at converting visitors versus the existing version of the site.

Now that such testing regimes are commonplace in the enterprise, it is inevitable that every team eventually encounters the egregeous and exhausting existentialist crisis that is test management. Gathering metrics for a single test is one thing, but what happens when the entire enterprise is pushing tests across thousands of sites all the time?

Claire Vo is Silicon Valley success story: she sold her startup Experiment Engine to Optimizely in 2017. Her particular winning formula was to help solve this exact problem for enterprises: Managing experiments at scale across thousands of sites, and measuring results in order to effect actionable changes overall.