SEO a/b testing is a controlled experiment: you change one variable on a set of URLs and measure organic impact against an untouched control set. The variable is usually a title tag, meta description, H1, hero copy, or internal-link block. The measurement is clicks, impressions, CTR, or conversions over a defined window from Google Search Console.
The misleading claim I hear constantly: "We A/B tested your page and it ranks better now." That is a rewrite, not a test. A real test needs a control group and a variant group, run at the same time, on comparable pages. Without a control you cannot separate your change from a Google update, a seasonal swing, or a competitor slipping.
Here is the playbook I use with Pro clients. Boring and repeatable, and it works.
Not everything is worth testing on a 30-page contractor site. Spend your statistical budget where the lift is biggest.
Title tags are the highest-impact test on small sites by a wide margin. The title controls CTR from the SERP, which drives every downstream number. Worth testing: city placement, brand placement (Brand | KW versus KW | Brand), benefit modifiers like Same-Day or Licensed, and character length (50 vs 60).
Meta descriptions have lower impact because Google rewrites them on roughly 70 percent of queries, based on my own logs. Still worth testing with a strong call-to-action or a number-led line. Do not test meta on a 5-page site. You will never reach significance.
Hero content means the H1 and the first paragraph above the fold. Try question H1 versus statement H1, or problem-led versus outcome-led opening. A tight hero rewrite moved Or's "garage door spring repair Denver" page from position 7 to 4 over six weeks.
Internal link blocks. Link experiments pay off especially well on landing pages. Test placement, anchor variety, and count (3 links versus 6). I have seen 12 to 18 percent click lifts just from moving a related-services block higher.
One trap: treating schema markup as an A/B variable. Schema either parses or it does not, and rich results show or they do not. That is a coin flip, not a test.
SearchPilot is the enterprise standard. Server-side traffic splits, Bayesian significance, a real confidence read per test. The catch is the price: five figures a month, built for sites doing 500,000+ organic sessions. Overkill for a contractor doing 4,000.
Google Optimize sunsetted September 30, 2023. It is gone. Anyone selling an "Optimize-based SEO test" in 2026 is reading an old script.
RankSense sits in the middle. It uses Cloudflare Workers to inject changes at the edge and split-test titles and meta at scale, priced $200 to $800 per month. A real option for sites doing 20,000+ monthly sessions already on Cloudflare.
SEOTesting.com is the lightweight option more agencies are adopting. It connects to Search Console, logs a change, and computes a before-and-after with a confidence band. Not a true concurrent A/B platform, but honest and useful for small sites at $30 to $70 per month.
The DIY stack I run for Starter and Pro clients: Search Console API for raw data, a Python script running a two-proportion z-test on CTR, and a Google Sheet logging every test. Cost: zero. Time per test: about three hours. HouseCall SEO layers this with AI visibility tracking, so the same data feeds both the test log and the AI citation report.
This is where most "SEO tests" fall apart. A 12 percent CTR lift sounds great until you learn it ran on 4 pages with 60 impressions each.
Honest significance has three requirements. First, enough impressions to detect the lift you care about: roughly 3,000 to 5,000 per cohort for a 10 percent CTR change, 12,000+ for a 5 percent change. Second, a window long enough to wash out day-of-week variance and ranking jitter. I never call a test before 21 days. Third, a control cohort that is genuinely comparable in volume, position bands, and query intent.
The math is a two-proportion z-test on CTR. Threshold: 90 percent confidence for landing page split testing on small sites, 95 percent for anything rolling out site-wide. Below 90 percent, the result goes in the inconclusive pile. About 40 percent of my tests land there, and that is normal. Anyone claiming 80 percent of their tests "win" is either at enterprise scale or misreading the math.
A few things from client work over the last 12 months that do not show up at conferences:
SEMrush and Ahrefs data is not built for SEO A/B testing on small sites. Volume numbers are inflated. Ranking signals are sampled, not measured. Surfer and Clearscope push every competitor toward one template, which is why every roofing page on page one reads identically now. On a contractor site, the only source I trust is Search Console pulled raw via the API.
Review count is overrated, and split-testing the review call-out proves it. Or's site has 13 Google reviews and beats a 253-review competitor for "garage door spring repair Denver." In February we swapped the block from "13 verified Google reviews" to "Read what 13 Denver homeowners wrote." The variant lifted contact-form submissions 22 percent versus 4 percent on the control over 28 days. The number was never the problem. The framing was.
AI content tools produce variants that fail tests. I ran two head-to-head landing page split testing experiments: one cohort got AI-only copy, the other got copy I wrote after a 30-minute owner interview. Owner-interview copy won both times, by 14 to 31 percent on form submissions. The same pattern held with how we rank chiropractor businesses: specific copy consistently outperforms generic. AI is fine for a first draft, not as the final variant in a real test.
| Tier | Monthly | Testing cadence | Tools |
|---|---|---|---|
| Starter | $750 | One test per quarter (titles or hero only) | DIY stack (Search Console API + z-test script) |
| Pro | $1,500 | One test per month, rotating titles, meta, hero, links, FAQ | SEOTesting.com or DIY stack |
| Custom | $3,000+ | Two tests per month, aggressive variable matrix | Optional RankSense (billed separately) |
Starter fits sites under 5,000 monthly sessions, where more frequent testing is statistical theater. Pro is where seo split testing compounds because cadence matches data volume. Custom fits sites doing 25,000+ monthly sessions or multi-location chains. Full breakdown in our SEO packages.
I do not sell SearchPilot subscriptions to contractor clients. The price does not fit and the platform is built for traffic profiles contractors do not have. If a competing agency told you they "use SearchPilot for your site," ask to see the contract terms.
Per test: 14 days for Google to recrawl and re-render variant pages, then a 21 to 28 day measurement window. Total: 35 to 42 days from change to verdict, every time.
Per engagement: real compounding revenue lift starts at month 4 to 6. Months 1 and 2 are the audit and first test. Months 3 and 4 are the first verdicts and rollouts. Months 5 and 6 are when rolled-out winners show in form submissions and booked appointments.
I lost a locksmith client in Atlanta last year because he left in month 3, before the lift appeared. Or in Denver stayed through month 6 and is up 41 percent year-over-year on form submissions. It just needs the patience most owners ran out of after years of unmeasured agency promises.
It is a controlled experiment where you change one element on a set of pages, hold an identical control cohort untouched, and measure organic impact over a 21 to 28 day window. Without a control it is not a test, it is a rewrite with hope attached. The change is usually a title, meta, hero, or internal-link block, measured with Search Console.
You pair similar pages by traffic and intent, change one variable on half, and leave the rest as the control. After Google recrawls, you compare the variant cohort's CTR change against the control's over the same window. The math is a two-proportion z-test at 90 percent confidence. On a 30-page site you run one to two tests per month, not ten.
At HouseCall SEO it is bundled into three packages: Starter at $750 with one test per quarter, Pro at $1,500 with one test per month, and Custom at $3,000+ with two. Enterprise platforms like SearchPilot run five figures a month. RankSense runs $200 to $800. SEOTesting.com runs $30 to $70 and pairs well with Starter.
Per test: 35 to 42 days from change to verdict, split into 14 days for indexing and 21 to 28 for measurement. Per engagement: real compounding revenue lift starts in month 4 to 6. Anyone promising seo split testing wins in 30 days is running tests too short to be valid, or rewriting pages and calling it a test.
For enterprise sites, look at SearchPilot's in-house team and the few agencies certified on their platform. For service businesses under 30,000 monthly sessions, find a boutique shop that runs the DIY stack honestly, publishes the test math, and names clients with live URLs you can verify. That is what I built HouseCall SEO to do. Or in Denver, Tomer at pinegaragedoors.com, and Alex at acelocksmithsf.com are real clients with real test logs. Book a free SEO consultation and I will walk you through them.
You run a US home-service business and you are tired of agency promises that never get measured. I will run an honest first SEO A/B test on your site and show you the math. Book a free SEO consultation with Lior and we will look at your top 10 pages together.
I specialize in home services SEO – taking websites that sit invisible on page three and turning them into the business Google and ChatGPT recommend first. I started on the developer side, writing software and doing SEO on the side, until I saw how much home-service owners were overpaying for work that quietly hurt them. So I built a method that fixes the broken technical work and the outdated thinking behind it.
From garage door companies to plumbers, roofers, locksmiths and cleaning services, the playbook is the same: rank where your customers actually search, earn real reviews, and back it with a fast site that books the job. No PBNs, no bought reviews, no directory spam – only work that survives Google’s next five updates. See exactly how it’s priced on the pricing page.
Send me your site and I’ll send back a free audit: what’s broken, what it’s costing you in calls, and the first three fixes.
No spam and no sales pitch. Just a clear look at what’s leaking leads.
