Report the median when at least three runs are valid.
PROTOCOL 2.0 · PUBLISHED JULY 16, 2026
Shared rules for every new benchmark
Measure the same way. Compare only what is truly comparable.
This protocol governs sample selection, website testing, manual review, quality control and public reporting for Web Respawn’s next local service-business benchmarks.
What changed
Version 1.0 showed the questions. Protocol 2.0 makes the answers more reproducible.
The published Lake County report stays intact as Version 1.0. New comparisons use these stronger rules and are labeled separately.
JSON, timestamps, versions and checksums stay in the private evidence archive.
Visible actions and service clarity are reviewed at 390 × 844.
A bounded crawl covers the homepage and as many as nine internal pages.
Only 404 and 410 responses count as confirmed broken after recheck.
Why preserve Version 1.0? Replacing old figures with a new method would hide the change. The original report remains a valid record of its stated collection window and rules.
Sample architecture
Balanced cells create fair comparisons—not a county census.
Every result is described as “in this sample.” Equal industry quotas help comparison but do not estimate the real mix of local businesses.
2counties×10industries×10sites=200websites
4counties×6industries×10sites=240websites
- 01PREDECLARE
Five local zones
Set the geographic frame before seeing website scores.
- 02DISCOVER
Record public searches
Log date, query, approximate location and discovery source.
- 03VERIFY
Prove local eligibility
Use the official site and, when needed, a chamber or directory record.
- 04DEDUPLICATE
One business and domain
Exclude directories, social-only profiles and unsupported national pages.
- 05RANDOMIZE
Use a recorded seed
Select two sites per zone and keep ordered reserves.
- 06LOCK
No healthy replacements
Unavailable sites remain unavailable after the sample is frozen.
Candidate-scarcity rule: the standard is two selected websites plus two ordered reserves in every category × zone cell. If an exhaustive pre-audit search finds only three strictly eligible sites, the cell may keep two selected sites and one reserve only when the shortfall is recorded before testing. Eligibility is never relaxed and no score may influence the exception.
Frozen definitions
Each measure has one pass rule—and one stated limit.
Unavailable, blocked, uncertain and insufficient observations remain separate from failures.
Final HTTPS
Final resolved URL uses HTTPS and completes TLS.
Not a security audit.HTTP upgrade
The HTTP version reaches HTTPS within five redirects.
Reported separately from final HTTPS.Mobile performance
Median of five PSI mobile Lighthouse runs; at least three must be valid.
Lab data, not real-user experience.Real-user Core Web Vitals
Eligible mobile origin-level CrUX data reports good p75 LCP, INP and CLS.
No CrUX data stays unavailable.Confirmed broken link
A bounded crawl finds a 404 or 410 after the retry rule.
Blocks, timeouts and 5xx stay uncertain.Visible primary action
A distinct call, contact, quote, estimate, book or schedule control is visible before scrolling at 390 × 844.
“Learn more” does not pass.One-interaction contact path
One interaction or less reaches phone, email, contact form or booking.
Forms are never submitted.Primary service clarity
The initial mobile viewport visibly names the main service category.
A logo or image alone does not pass.Geographic clarity
The homepage visibly names a real municipality, county, region or service area.
No inference from domain names.Service detail path
One homepage interaction reaches a substantive service page or section.
A bare list of names does not pass.Technical indexability
Usable 200 HTML, not robots-blocked and no noindex directive.
This does not prove Google indexing.Bounded link review
Wide enough to find real friction. Narrow enough to be respectful.
1homepage
→≤9internal pages
→≤100destinations
→404 / 410confirmed broken
- Breadth-first crawl with navigation, service and contact pages prioritized.
- Same-domain, HTML-like destinations only; no forms, authentication or action URLs.
- Robots directives and rate limits are respected; redirects and retry outcomes are recorded.
- 403, 429, 5xx, timeouts, TLS errors and bot blocks are retried once after 24–48 hours, then classified as uncertain—not broken.
- Zero eligible links means “insufficient link sample,” not a clean pass.
Quality control
The evidence is checked before the headline is written.
Keep every raw PSI response and require at least three valid mobile runs.
Independently recode at least one fifth of rendered mobile fields.
Clarify or remove a field when reviewer agreement is too low.
Show raw counts, denominators and unavailable states beside every rate.
Analysis and claim boundaries
Show uncertainty. Avoid a winner. Never turn a proxy into a promise.
- Counts and eligible denominators
- Wilson 95% intervals for rates
- Median and IQR for continuous measures
- Absolute percentage-point differences
- Accessible data behind every chart
- One county’s businesses are “better”
- A website feature caused leads or rankings
- Technical indexability means Google indexed it
- Lighthouse lab data equals real-user Core Web Vitals
- An automated score proves accessibility compliance
Publication threshold: headline a difference only when it is at least 10 percentage points or clearly important in practice. Do not publish municipality rankings or county-by-industry conclusions based on cells of 10.
Evidence retention
Public anonymity. Private reproducibility.
The public dataset never includes business names, domains, screenshots or quoted copy. The private evidence archive retains enough information to review the work without using audited companies as sales examples.
PRIVATE CANDIDATE LOG+FROZEN SAMPLE+RAW TEST OUTPUT+REVIEW NOTES+CHECKSUMS→ANONYMOUS PUBLIC DATA
Research series
The rules are public before the next county result.
That order matters: the method should shape the finding, not be rewritten to support it.