What are some of the best and worse experience optimization products build with A/B testing at the core? Why?

There is so much everyone is building and the volume is going to increase only, if anything. In the old days, optimisation was about copy, buttons, thumbnails but now entire experiences can be optimized. What are the offerings that make sense? How would you conceptualize a product around this?

maguay's avatar
a month ago

The one way I use A/B testing regularly is in emails—and it's been a helpful way to test copy and see subjects people are interested in. But at the same time, it's a potentially dangerous thing. A/B testing headlines—and the general goal of getting people to click on links—is part of the reason clickbait is such a problem. You can run the test and find that one bit of text converts to clicks better, and so it's better for that goal, but that doesn't absolutely mean it's the best copy.

In the same vein, I'm sure many things in products we use daily that feel frustrating, from popovers asking for email sign in to Facebook's apps moving buttons around often, do result in people tapping things or filling in forms that their teams are incentivized to get users to do. So the test delivered a win, and yet users end up feeling frustrated as they have to re-learn their way around and interface or get interrupted while they're trying to accomplish something in an app.

As such, I think A/B tests can be helpful, but you need to very carefully define what success means. A raw higher number of clicks or pageviews or signups might not be the win it seems to be if it's also hurting some other metric, or making the user experience somewhat worse in ways numbers might not show until you've already lost user trust.

1 point
What's your favorite support ticketing platform?

We used to use Zendesk, so I'm somewhat familiar, but wondering if anything cool has been launched in this space recently. Table stakes would be ability to assign tickets, add status, and have back...

What's the best way to let people choose an event time and add it to their calendar?

I'm looking for a way to list 3 weekly onboarding events where each one is run by a different person on our team, and where multiple people can attend each events. Ideally we could let people see a...

What budgeting tools do you use?

Hi everyone, I have an idea for a new budgeting app, but wanted to get your thoughts on the tools you use. What budgeting tools do you use? What do you LOVE about them? What’s stopping them fr...

The community for power users.