Website Security Regression Checker

Compare two website security audits and see exactly which controls improved, stayed the same or regressed, with the severity change for each one.

Runs locally

Everything happens in your browser. What you paste or drop here is never uploaded, logged or stored.

Baseline audit (JSON)

Or drop the baseline JSON

Read locally.

Current audit (JSON)

Or drop the current JSON

Read locally.

A finding that vanished was not necessarily fixed

This is the entire difficulty of comparing two security audits, and it is why most comparisons are quietly wrong. A finding present last month and absent today has two completely different explanations: somebody fixed it, or the check never ran.

Reporting the second as the first tells a team that a problem went away while nobody touched it - and nobody investigates a resolved finding. The error always points the same way, towards reassurance, which is the one direction a security tool must not fail in.

So an absence is only called Resolved when the current audit demonstrably still exercised that area. When it did not, the finding is reported as Not re-tested, counted separately, and explicitly excluded from the resolved total.

Findings are matched on a stable identifier

Every check in the audit carries a rule identifier that does not change with its outcome. The HSTS check reports the same rule whether it passes, fails or times out. That is what makes a comparison possible at all: a control that broke shows up as a severity change on one row, rather than as one finding disappearing and an unrelated one appearing.

Checks that report more than once - one row per cookie, one per policy warning - derive their identifier from the subject, using a fixed hash rather than a runtime one. The distinction matters more than it sounds: a hash that changes between processes would make every audit after a restart look like a completely new report.

The outcomes it reports

Regressed - present in both, worse now.
New - not in the baseline at all.
No longer verifiable - had a definite result before and does not now. Not an improvement, whatever the ordering suggests: the control stopped being observable.
Not re-tested - absent because the check did not run. State genuinely unknown.
Improved, Resolved and Unchanged.

If the two documents were produced under different scoring methodology versions, or turn out to be audits of two different sites, that is stated at the top rather than silently absorbed - individual findings still compare correctly, but the two scores were made with different rulers.

Both documents stay in your browser

Two audit reports for the same site, side by side, are a record of what was wrong and what still is. There is no reason for that to leave your machine, and a diff of two JSON documents needs no server.

Frequently asked questions

What do I paste in?

Two exports from the Website Security Audit, of the same site, taken at different times. Schema version 1.x, including older exports made before schema versioning, which are upgraded on read.

Why is a finding listed as "Not re-tested" rather than resolved?

Because it is absent from the current audit and nothing else in its category reported either - the pattern of a check that did not complete. Calling that a fix would be inventing one.

Can I compare two different sites?

It will compute the comparison and warn you first. A difference between two sites is a difference, not a regression - the Score Comparator is the right tool for that.

Why did a finding move to "No longer verifiable"?

Its check used to return a definite answer and now returns Unknown. The configuration may be untouched, but your ability to confirm it is not - so it is grouped with the regressions rather than with the improvements.

Are the reports uploaded?

No. Both are read and compared in your browser, and the exports are generated there too.

References

OWASP Web Security Testing Guide

Related tools

What this tool can and cannot tell you. It reports what is present in the input you provide. It cannot see anything you did not give it, and a clean result means nothing was found in that input - not that the wider system is secure.

Security guidance here follows current published sources - OWASP, MDN, the relevant RFCs, NIST, CISA, FIRST and MITRE - which are linked beside the specific claims they support.