Skip to content
Blog/Notes from the edge
Litmus 8 min read

What boilerplate_ratio Counts, and What It Cannot See

NP
NanoParse team

Litmus returns twenty signal groups in the same response body as the markdown. Two of them look like verdicts on the page: boilerplate_ratio and content_density. Both names promise more than the code delivers, and a developer ranking sources by either will rank them wrong.

Fourteen URLs were submitted to the endpoint on 2026-10-31, across six page classes. Eleven completed, three were refused as blocked. Every payload is committed beside this post, and every number below reproduces from those files — except the word counts on pages that rotate their own content, which moved by a few words on a re-run an hour later while every ratio stayed identical.

The result, sorted by boilerplate_ratio:

ratio density words content_type url 0.67 0.73 1510 documentation kubernetes.io/docs/concepts/overview/ 0.57 0.93 470 tutorial developer.mozilla.org/.../Fetch_API 0.27 0.71 555 tutorial usa.gov/ 0.15 0.96 533 academic_paper en.wikipedia.org/wiki/Boilerplate_(text) 0.09 0.86 4156 tutorial en.wikipedia.org/wiki/Web_scraping 0.05 0.70 3852 academic_paper docs.python.org/3/library/json.html 0.02 1.00 1311 news_article bbc.com/news 0.01 0.81 1375 product_page stripe.com 0.00 0.50 135 general example.com 0.00 1.00 949 blog_post news.ycombinator.com 0.00 0.79 554 documentation docs.anthropic.com/en/docs/overview

Read as a quality ranking, that table says the Kubernetes documentation page is two-thirds junk and the Hacker News front page is perfectly clean. Both readings are wrong, and the reasons are in twenty lines of source.

What the code counts

Three selectors, one denominator:

const navLen = ($("nav").text() || "").length; const footerLen = ($("footer").text() || "").length; const cookieLen = ($('[class*="cookie"], [id*="cookie"], [class*="consent"], [id*="consent"]').text() || "").length; const totalLen = ($("body").text() || "").length || 1; const boilerLen = navLen + footerLen + cookieLen; const ratio = Math.round((boilerLen / totalLen) * 100) / 100; return { ratio: Math.min(1, ratio), confidence: "high" };

Percentage of all page text that sits inside a

Litmus1,660 words

Test the page your agent just read.

Clean Markdown and 20 trust signals, $0.002 per parse.

Run a test

Your agent deserves better data.

Clean Markdown and twenty deterministic trust signals, priced per parse.

10 free parses a month · no account · no API key