Split Testing Is a Waste of Time

February 10, 2026

Most brands I come across aren’t at the size where they’ll get any meaningful, statistically significant results from most A/B testing.

If you’re only sending each campaign to a couple thousand people, and the open rate varies by a few percentage points, it’s hard to draw too many conclusions from that data.

There’s no harm in setting up tests on subject lines for your welcome email and other core emails that receive a lot of traffic — and I always do this. But there’s not much point in reading too much into results from campaigns until you’re sending at a large enough volume.

The pop-up form is another area that we always test. It typically receives the most interactions of any aspect of your email system, and even small improvements can dramatically increase your overall email performance.

At the end of the day, don’t worry about setting up A/B tests until you’ve got all the fundamentals in place.