A hundred sites, not one
One board for the whole portfolio: what got worse, where, and since when. Built for people running dozens of projects, not for auditing one and closing the window.
Command line · Source is open
What changed on your sites since last week
Technical auditing for people who run many sites, not one. Compare crawls, catch what a deploy broke, and work through hundreds of thousands of URLs without bringing the machine to its knees.
Every two weeks: measurements, defects, worked examples. Nothing else.
$ crawlforge diff example.com27 Jul → 3 Aug · 3,348 URLs
Got better
+11HTTP-404-INTERNAL resolved
404 → 301 /googlechrome
404 → 301 /internetexplorer
Got worse
−1HTTP-REDIRECT-CHAIN new
/googlechrome → article → mobile
A crawl is a photograph. Your job is the film.
You audit a site, hand over the report, changes ship. Two weeks later you crawl again and there are another thousand rows in front of you, with no way to tell which ones are new.
Now multiply that by every site you look after.
Three things you do by hand today
Compare two crawls
What was resolved and what appeared after the last deploy. In your CI, it fails the build when something serious gets in.
No Java, no browser
Written in Rust. A table of two hundred thousand rows scrolls like your system's, because it is your system's.
Open source, and specific about which parts
A tool that asks for your staging credentials should be readable. And a rule that decides whether a client's page is indexable should be arguable. So the engine is open — and the app is what the subscription pays for.
Apache 2.0 · out now
Free and open, today
- The crawler and its 63 audit rules
- The command line: crawl, audit, diff, export
- WordPress and Astro adapters
- Every crawl is a plain SQLite file. Query it yourself.
Being considered
Where this could go
- Native apps for macOS, Windows and Linux
- A portfolio board across all your sites
- Scheduled crawls and change alerts
- Sync between machines and team members
No dates promised. A native interface is a lot of work, and the command line comes first.
Back the project
The open part has no price and no plans to get one. If it saves you an afternoon a month, a recurring contribution keeps the rules catalogue growing.
Become a backer →You choose the amount. Cancel whenever.
Sponsor as a company
Agencies and hosts that run this across a portfolio can sponsor the work at a level that matches what it replaces, with your name on the repository.
Talk to us →Invoiced, with VAT number.
○ macOS · thinking about it○ Windows · thinking about it● Command line · all three, today
The numbers, with their conditions
If you are going to audit with this, check how it was measured first. The method sits next to every figure.
487,621
URLs in a single crawl, with memory flat
A news site with fifteen years of archive. Memory follows the pending queue, not the size of the site.
0 / 1,800
extraction differences against an established crawler
The same 300 URLs to both tools. Status, title, meta description, H1, canonical and indexability.
63
audit rules, each with its own test case
The rules are the product: when one is wrong, you stop trusting the whole report.
Built in the open
Read all posts →26 Aug 2026Measurements
Four rules, a port, and a 24% that nearly slipped through
0.10.0 brings the rules that were missing to judge what the crawler already knew, and changes what counts as your own site. The interesting part is that performance dropped 24%, the regression test let it pass, and the first explanation I came up with was wrong.
26 Aug 2026It brokeThe broken link that fell between two conditions
0.9.0 started checking where external redirects end up. The view that answers "which of my links are broken" still could not find them, because a 301 is not broken and nothing links the 404 at the end. Recording the data and showing it are two different jobs, and this is the second one.
26 Aug 2026It brokeThe test passed with the bug in place
0.9.2 fixes something small: a crawl waiting on other people's servers no longer looks dead. What is worth telling is that I wrote the test, watched it go green, reverted the fix to make sure it would fail, and it stayed green. The test was covering the part that was not broken.
The code is out. Every two weeks for the rest.
The work stays visible: what gets measured, what breaks, and how a real audit is run — with the commands and the numbers, on sites that exist. No download required to read any of it.