What we measure, exactly
A benchmark is only as good as its definitions. These are the ones behind every number on this site, in the form question → rule → source → date.
Which listings are read?
Google Maps results for the query “personal injury lawyer <metro>”, scrolled until Google renders no more — the whole market, not a sample. Rank is kept for each listing.
markets.csv · listings_rendered · 2026-08-26
Which firms are measured?
Listings with at least 40 reviews whose Reviews tab could be read. Under 40 reviews a reply rate is noise.
min_reviews_per_firm = 40
What counts as “unanswered”?
A review with no owner reply visible underneath at the moment of reading. A reply given by phone or email is real and cannot be observed; it is not counted.
data-dictionary.csv · reviews_unanswered
Which reviews are read per firm?
The 50 most recent, after sorting the Reviews tab by “Newest”. Without that sort Google shows “Most relevant”, and “your last 50 reviews” would be a false sentence.
avis.py · sort newest · verified 30/30, 28/30, 20/20, 50/50 on 4 firms
What is a “silent” firm?
A measured firm where fewer than 30% of the reviews read carry a visible owner reply.
firms_under_30pct_reply
Why is no per-city rate published for the cohort?
Four firms per city is too small a cell. No rate is published for any cell under 10 firms, to avoid re-identification and false precision.
small_cell_rule · Perplexity/Gemini panel 2026-08-27
How is a firm's local percentile computed?
From the whole-market cohort: every measurable firm of the metro (40 reviews or more, Reviews tab readable), its 50 most recent reviews, sorted newest first. A firm's number is the share of those 50 with no visible owner reply; its percentile is the share of the metro's firms that leave a smaller share unanswered. It is published only when the metro has 20 firms or more in the cohort, and never for a firm named in public.
machine/benchmark.py · release 2026-08 v2.0, markets-cohort.csv
Why does the whole-market cohort look so different from the first cohort?
The first release read the 20 most silent firms of five metros, chosen for outreach: 89% of their recent reviews had no reply. The whole-market cohort reads every measurable firm: the median firm leaves about one recent review in twelve unanswered. Both numbers are true; they answer different questions. The page says which one it is showing each time.
release 2026-08 v2.0 · firms_cohort vs market_cohort in benchmark.json
What is excluded?
Listings under 40 reviews; listings whose panel did not render after three attempts (a listing that renders late is not an absent listing — 9 of 20 came back on the second pass); national chains are flagged, not removed.
benchmark.json · exclusion
What is checked by hand?
At least 10% of firm rows per release are re-read manually. On 2026-08-26 the review sort was re-verified on four firms and the rank bias of the scroll was measured (rates by rank band: 12% / 25% / 12% / 0%).
benchmark.json · manual_check
What is a release?
A frozen folder: markets.csv, firms-recent-reviews.csv, data-dictionary.csv, benchmark.json, CITATION.cff, SHA256. It is never rewritten; a correction is a new version (v1.1) with a changelog line.
data/CHANGELOG.md
Who owns the data?
Published under CC BY 4.0: reuse, including commercial, with attribution. Cite the release identifier.
creativecommons.org/licenses/by/4.0
Whether an unanswered review costs a firm a case. That is not observable from outside and we do not estimate it.
Whether a firm replied privately. Only a visible reply is counted.
The method is reviewed by us, not yet by a named ethics attorney; the state-rules pages will be published only after that review.