A/B testing Flow2 with PostHog

A/B Testing Onboarding and the Paywall

PostHog demo

One guy, one phone, and PostHog.

Just launched Flow2

Where I'm at

A mobile presentation and short-form video creation app. Build it on your phone, film yourself over it, post it.

I've been going slow on purpose. The whole job right now is getting onboarding and the paywall right.

PostHog runs the experiments. Traffic is limited, so the numbers are small. The lessons aren't.

PostHog runs my analytics

Setup

iOS SDK, feature flags, experiments, SQL. One tool.

Separate mobile project. My own testing and TestFlight builds are filtered out.

Every event knows which arm the user is in, so I can slice the whole funnel by arm in one query.

Using Claude Code and PostHog MCP to set up most of this.

What are we testing?

What we tested

A: The onboarding story. The normal version vs our mascot telling it around a campfire.

B: Creation step. Three tap-to-start ideas vs a brief we write for you, with one Build button.

The paywall wasn't an A/B test. I changed it three times and tracked where each one was shown.

The campfire story didn't help

Experiment A

0 lift, anywhere A little worse in a couple of spots. Two good ideas, zero lift. Still a result: I stopped polishing what wasn't broken.

The brief changed nothing

Experiment B

3 in 4 submit a topic, either version Same number for both. The thing I built to fix this didn't fix it.

The real problem was somewhere else

The real leak

1 in 3 leave while the flow builds (15 to 30 seconds)

1 in 4 get to the finished flow and actually save a video

1 in 6 bail before the first screen even picks a variant

People wanted it. They didn't want to pay today.

The paywall

Paywall after signup: about 6 in 10 hit Buy

About 7 in 10 of those backed out at Apple's sheet within a few seconds

Paywall on skip: everyone backed out. Zero sales. I killed it

Launch reminder: mostly dismissed, a sale now and then. Costs nothing, so it stays

People wanted it. They didn't want to pay today.

What I'd tell myself on day one

Lessons

My experiment’s first step fired before the flag did, so PostHog ignored most of my users. Check that on day one.

With small traffic, the experiment tells you if the idea worked. The funnel tells you where people actually leave. Put the variant on every event and you get both from the same data.

Tag every paywall with where it showed up. I changed pricing on the server, shipped nothing, and could still see exactly what happened.

What's next

Keep the presentation build running in the background and ping people when it's ready. The wait shouldn't be the exit.

Fix the story metric so that test can actually finish. Keep both running.

The next test goes on the wait, not on the story.

Built in Flow2. Measured in PostHog.

Thanks

This whole deck is a Flow2 flow. Brent Brookler, flow2.co

Try Flow2