A/B Testing Candidate Outreach: What to Test First

A/B testing candidate outreach works best when you test in priority order: the opening line first, then message length, then the call to action, and timing last. Most recruiters test the lowest-impact variable (send time) and ignore the highest-impact one (the opener), then wonder why their numbers don't move. This article gives you the right test order, a practical framework, and copy-paste variants to start with.

TL;DR

• Test in order of impact: opener → length → CTA → timing.
• Change one variable at a time, or you won't know what moved the result.
• Brevity is a high-confidence lever: under 400 characters earns a 22% response lift (LinkedIn, 2024).
• Timing is the weakest lever — weekday send-time barely moves the number, so test it last.
• Measure qualified reply rate, not raw response rate, so you don't optimize toward "no thanks."

What should you A/B test in candidate outreach first?

Test the opening line first, because it carries more of the result than anything else in the message. The opener is what a candidate reads in the mobile preview before deciding to open, so it disproportionately determines your reply rate — which makes it the highest-return thing to experiment with.

The common error is inverting this. Recruiters often start by testing send times or subject-line emoji because those are easy to vary, but those are marginal levers. You can perfect your send time and still get ignored if the first line reads like every other recruiter message. Start where the leverage is: rewrite the opener (more on what makes one work in the InMail opening line), prove it moves qualified replies, then work down the list.

The test priority order

Test variables in descending order of impact so your early experiments produce the biggest gains. Here's the order and why each sits where it does.

Work top to bottom. There's no point fine-tuning your CTA if your opener is losing people before they reach it. And testing timing before message quality is optimizing the wrong end of the funnel entirely.

How do you run a valid A/B test on outreach?

You run a valid test by changing exactly one variable, splitting your candidates randomly, and giving each variant a large enough sample to trust the result. Break any of those three and the test tells you nothing — you'll just be reading noise.

Three rules keep a test honest:

• One variable at a time. If you change the opener and the CTA, a lift could come from either. Isolate the change.
• Random, comparable splits. Don't test variant A on senior engineers and variant B on juniors — the audience difference will swamp the message difference.
• Enough volume. Small samples swing wildly. A 2-of-10 vs 3-of-10 result isn't a finding; it's a coin flip. Wait for a meaningful number of sends per variant before declaring a winner.

One more discipline: measure the right outcome. Optimize for qualified reply rate — positive, interested responses — not raw response rate, or you risk crowning a variant that simply provokes more "not interested" replies. We cover that distinction in recruiting outreach metrics.

Copy-paste: a test plan and starter variants

Here's a ready test plan you can run this week, plus paired variants for the top three levers. Run one pairing at a time.

Test plan (copy into a doc):

Hypothesis: [e.g., a project-specific opener beats a generic one] Variable tested: [opener / length / CTA — one only] Variant A: [control] Variant B: [change] Sample: [N per variant — keep it meaningful] Audience: [same role/seniority for both] Window: [1 week; 90% of replies land within 7 days] Metric: qualified reply rate Winner: [decide only after the window closes]

Opener test (priority 1):

A (generic): "Hi [Name], I came across your profile and was impressed by your background." B (specific): "Hi [Name] — your write-up on [specific thing] stuck with me, the [detail] especially."

Length test (priority 2):

A (long): full role description, company background, benefits in the first message. B (short): one-line hook + one-line role + ask, under 400 characters, details behind a link.

CTA test (priority 3):

A (heavy): "Please send your resume and availability for a call." B (light): "Open to 15 minutes this week, or is the timing off?"

Start with the opener test — it's where the biggest, fastest gains usually are. Tools help here too: Everyjob's Advanced Analytics lets you generate opener variants and track which one wins on qualified reply rate, so the test-and-learn loop closes without a manual spreadsheet.

What's not worth testing (yet)?

It's not worth testing micro-variables — emoji, exact send minute, sign-off wording — until your opener, length, and CTA are dialed in. These low-impact tweaks produce small, noisy effects that are easy to mistake for signal, and they distract from the levers that actually move the number.

Trigger-based timing — reaching out right after a role change or company event — is often cited as a strong lift (some practitioners report meaningful gains), and it's worth building into your sourcing. But note it's a targeting decision, not a message A/B test; you're changing who you contact and when, not which of two messages wins. Keep it in your strategy, just don't confuse it with a message experiment. Send-time is similar: weekday timing barely moves results, so it sits at the very bottom of the test queue.

Frequently Asked Questions

What should you A/B test in recruiting outreach?

Test in priority order: the opening line first, then message length, then the call to action, and timing last. The opener decides whether the message is read at all, so it has the highest impact and the best return on experimentation. Avoid starting with low-impact variables like send time or emoji — perfecting those won't help if the first line is losing candidates before they open the message.

How do you run an A/B test on candidate messages?

Change one variable at a time, split candidates randomly across comparable audiences, and give each variant enough volume to trust the result. Run the test over about a week, since 90% of replies arrive within seven days. Measure qualified reply rate rather than raw response rate, so you reward messages that generate genuine interest instead of ones that simply provoke more "not interested" replies.

How big does an outreach A/B test sample need to be?

Big enough that the result isn't a coin flip — small samples swing wildly, so a 2-of-10 versus 3-of-10 difference is noise, not a finding. There's no single magic number, but the more contacts per variant, the more you can trust the winner. If your sending volume is low, run the test longer rather than declaring a result early on a handful of replies.

What's the highest-impact thing to test in outreach?

The opening line. It appears in the mobile preview and determines whether a candidate opens and reads the rest, so it carries more of the result than length, CTA, or timing combined. Test a generic opener against a specific, candidate-referencing one first. Once the opener is optimized, move down to length and CTA, leaving send-time experiments for last.

Should you test send times for candidate outreach?

Test them last, if at all. Weekday send-time has only a small effect — Saturday and Friday underperform, but Monday through Thursday are roughly even (LinkedIn, 2024). Optimizing send time before fixing the message is tuning the weakest lever. Build sensible weekday sending into your process, then spend your testing energy on the opener, length, and CTA, which move the number far more.

Key Takeaways

• Test in order of impact: opener → length → CTA → timing.
• Change one variable at a time across comparable audiences.
• Brevity is a reliable lever — under 400 characters earns a 22% lift (LinkedIn, 2024).
• Measure qualified reply rate, not raw response, to avoid optimizing toward rejections.
• Don't sweat send-time or emoji until the message itself is dialed in.