All guides
7 min read·Intermediate·5 steps·2 screenshotsNumbers and testing

Subject line A/B test: find out what people actually open

The subject line decides the open, the open decides everything else. An A/B test is the cheapest way to stop arguing and start measuring.

People argue about subject lines for hours. A test settles it in a week: two variants, recipients split in half, the numbers pick the winner. This guide is about what to test, for how long and how to read the result.

1

Test one thing

Question vs statement. Company name in the subject vs without. Short vs longer. Not three changes at once, or you will not know which one worked.

colldly.com/dashboard › Campaigns
Setting the second subject line variant in a Colldly campaign
Setting the second subject line variant in a Colldly campaign
2

Enter variant B in the campaign

In the sequence, at the first email, click "Subject line A/B test" and enter the second variant. Recipients are split in half automatically; everything else (copy, timing, mailboxes) stays the same.

3

Wait for the sample

At least 100 companies contacted per variant. A 3% difference on 30 companies is one opened message, not a result. At 100 companies a day you have a result in two days.

Tip: Do not evaluate earlier than 48 hours after the last company was sent. Opens keep arriving for two days.
4

Read opens and replies

A subject line with more opens but fewer replies is lying: it triggered curiosity but did not promise what the email delivers. The winner is the variant with more replies; opens are just a hint.

colldly.com/dashboard › Statistics
Campaign statistics in Colldly comparing variants
Campaign statistics in Colldly comparing variants
5

Lock in the winner and test the next

The winner becomes variant A of the next campaign and you test a new idea against it. After five rounds you have subject lines proven to work in your market.

What to avoid
Evaluating after 20 emails. That is a coin toss.
Testing the subject line while also changing the copy. You will not know what worked.
Picking the winner by opens. Replies decide.
Checklist
  • One variable per test
  • Variant B entered in the campaign
  • At least 100 companies per variant
  • Evaluated 48 hours after the last send
  • Winner by replies
  • Result written down and used in the next campaign
Questions about this guide
Can I test the email body too?
One campaign tests the subject line. Test the body with two campaigns on the same list split in half and compare them in the statistics.
Which subject lines win most often?
Short, specific, with the company name or a detail from its website, no sales words. "The second lift at Kingsway Motors" beats "Partnership opportunity" almost every time.
Does the AI test subject lines by itself?
The AI writes a different subject from website facts for every company. The A/B test is useful with your own subject line or to compare the AI subject with yours.
Continue with
Next guide in order: Agencies: a client campaign from workspace to report
Start today

Colldly does all of this for you

It finds the companies, writes personalised emails, guards deliverability and puts the replies into one inbox. Start for free, no card.

Start for free
No credit card · First 200 credits free · Cancel anytime