CRO Testing: What to Test First (And What to Ignore)

Every CRO article tells you to test everything. That's terrible advice if you have under 100K monthly visitors.

I get it. You've read the blog posts. "Test your headlines, test your CTAs, test your button color, test your form fields, test your hero image, test your testimonials, test your pricing, test the shade of gray on your footer links." The CRO testing advice out there reads like someone got paid by the word.

Here's the problem: if you have 20,000 monthly visitors and a 3% conversion rate, you can run maybe 2-3 meaningful tests per month. That's 24-36 experiments per year. You cannot afford to waste a single one on something that doesn't move the needle.

I've watched teams burn entire quarters on low-impact tests while obvious conversion killers sat right on their homepage. This post is the prioritization framework I wish someone had given me before I started running website conversion optimization experiments.

The IEE framework: what to test first

Most people know the ICE framework (Impact, Confidence, Ease). I use a modified version I call IEE, because "confidence" is a vague feeling and I'd rather base decisions on data.

Impact: how much traffic touches this element?

If 100% of visitors see your headline but only 12% scroll to your testimonials section, the headline has roughly 8x the testing surface area. A 10% lift on an element everyone sees is worth far more than a 30% lift on something buried below the fold.

Map your page from top to bottom. Assign each element an approximate "visibility rate" based on your scroll depth data. If you don't have scroll data, install a heatmap tool for one week before you start your CRO testing program. You need this baseline.

Effort: can you test it in an hour or does it need a developer?

Copy changes are nearly free. You can swap a headline in minutes. Layout changes require design and dev work, QA across devices, and usually break something unexpected. A copy test that takes 30 minutes to launch will always beat a redesign that takes two sprints, even if the redesign has higher theoretical upside.

Evidence: is there data suggesting this is broken?

This is the one most teams skip entirely. Before you test anything, look at the data you already have:

• Bounce rate by page section -- where are people leaving?
• Click maps -- are people clicking on things that aren't buttons?
• Form analytics -- which field is causing the biggest drop-off?
• Session recordings -- watch 20 sessions. What confuses people?

Evidence turns "I think we should test this" into "the data shows 40% of users abandon the form at the phone number field, so we should test removing it." The second version wins at a dramatically higher rate.

What to test FIRST

These are the conversion rate optimization testing categories that consistently produce measurable results, even with modest traffic.

Headlines (27-104% lift potential)

I keep coming back to this because the data is overwhelming. In the experiments we've run through SplitMonk, headline changes produce the most reliably large effects of any single-element test.

A B2B SaaS client changed their headline from "The All-in-One Platform for Customer Success" to "Cut churn by 34% in 90 days" and saw a 47% increase in demo requests. That's not unusual. Generic benefit headlines almost always lose to specific, quantified outcomes.

Rules I follow for headline variants:

• Always test at least one variant with a specific number
• Always test at least one variant that names the pain point directly
• Never test two variants that are basically the same sentence rearranged

CTAs (text, color, and placement)

CTA copy matters more than CTA color. I'll say that again for the people in the back: what the button says matters more than what color it is.

"Start Free Trial" vs. "See My Results" isn't a minor copy tweak -- it's a fundamentally different promise. One says "here's a free thing," the other says "here's what you'll get." We consistently see 15-30% lifts from CTA copy changes.

Placement matters too. I've tested moving the primary CTA from below a feature list to above it and seen a 22% lift. The hypothesis is simple: people who are already convinced don't need to read four more paragraphs before they can click.

Social proof (34-42% lift)

Adding social proof where none exists is one of the closest things to a guaranteed win in CRO testing. But the type of social proof matters:

• Specific metrics ("12,847 teams use [Product]") beat vague claims ("Trusted by thousands") by 34-42% in tests I've run
• Logos near the CTA outperform logos in a dedicated section lower on the page
• Customer quotes with names and photos beat anonymous testimonials every time

The pattern is specificity. "We increased revenue by 23% in the first quarter" will always beat "Great product, highly recommend." If you have customers willing to share real numbers, that is the single most valuable CRO asset you own.

Form length (up to 120% lift from reduction)

If you have a form with more than 4 fields, test removing fields. This is almost always a win. One client reduced their demo request form from 8 fields to 3 (name, email, company) and saw signups increase by 120%. Yes, lead quality changed -- but the volume increase more than compensated.

The general rule: every additional form field reduces completion by 10-15%. If you're asking for phone number and it's not absolutely critical to your sales process, remove it. Test it and prove me wrong -- I bet you won't.

What to test LATER

These tests can produce results, but they require more traffic, more time, and more effort to execute properly. Save them for after you've exhausted the high-impact copy tests above.

Layout changes

Moving sections around, changing the visual hierarchy, adding or removing entire page blocks. These tests are harder to implement, harder to isolate (you're changing multiple things at once), and typically require significantly more traffic to reach statistical significance because the effect sizes tend to be smaller than copy changes.

Test layout changes when you've already optimized your copy and you're looking for the next tier of gains.

Pricing page experiments

Pricing tests are high-stakes and slow. They affect revenue directly, so you need to be very confident in your sample size. They also tend to have longer conversion cycles -- someone who sees your pricing page today might not convert for a week. That means longer test durations.

I'd rank pricing experiments as the second or third thing you optimize, not the first.

Multi-step funnels

Testing individual steps in a multi-step flow (onboarding wizards, checkout processes, multi-page signups) is important work but it compounds complexity. You need adequate traffic at each step, and upstream changes invalidate downstream results. Get your landing page converting first, then work inward.

What to NEVER waste traffic on

I'm going to be blunt. These tests have a near-zero chance of producing a statistically significant result for sites under 500K monthly visitors. They're not just low priority -- they're actively harmful because they burn traffic you could use on tests that matter.

Button color tests

Unless you have millions of monthly visitors, you will never reach significance on a button color test. The effect size is so small (typically 0-2%) that you need enormous sample sizes to detect it. The famous "red vs. green button" case study that launched a thousand blog posts? That was run on a site with tens of millions of visitors. If that's you, great. If not, stop testing button colors.

Font changes

Switching from one sans-serif to another slightly different sans-serif will not measurably change your conversion rate. I've never seen a font test produce a significant result on a site under 1M monthly visitors. If your font is genuinely hard to read, just fix it. You don't need a test for that.

Minor copy tweaks

"Get Started" vs. "Get Started Free" vs. "Get Started Now" -- these are not meaningfully different messages. You're testing word-level variations when you should be testing concept-level variations. "Get Started Free" vs. "See Your Dashboard in 60 Seconds" is a real test. The first three are noise.

How to build a testing roadmap

Here's the exact process I use to build a 90-day CRO testing roadmap:

Week 1: Audit. Install heatmaps and session recording. Review your analytics for bounce rates, exit rates, and conversion rates by page section. Watch 50 user sessions. Write down every moment of confusion, hesitation, or rage-clicking.

Week 2: Prioritize. Score every test idea on the IEE framework. Be honest about impact and evidence. If you're guessing about impact, you haven't done enough research. Pick the top 6-8 ideas.

Weeks 3-12: Execute. Run your tests in priority order. Each test gets a minimum of 7 days and must reach its calculated sample size before you call it. After each test, document the result and what you learned. Feed those learnings into new hypotheses.

This is exactly why we built SplitMonk to prioritize tests automatically -- it scores your page elements by visibility and conversion impact, then suggests which copy experiments to run first. But even without a tool, this framework works. You just need the discipline to follow it.

The bottom line

CRO testing is not about running more tests. It's about running the right tests. If you only have traffic for three experiments this month, make them count: test a radically different headline, test a CTA that makes a specific promise, and test adding quantified social proof near your conversion point.

Stop testing button colors. Start testing messages.