PXTA 0.3 Compares Two Audits and Shows What a Tag Manager Change Added

Most sites that run a tracking audit already know some of what it finds. A cookieless analytics ping legal signed off on, an old pixel on the blog nobody has got round to. What they need from the next audit is the difference: what this deploy, or this Friday’s tag manager publish, changed.

PXTA 0.3 adds pxta diff. Give it two findings.json files and it lists the findings that are new, the ones that are gone, the trackers that started or stopped sending data before consent, after a reject or after an accept, and anything that changed about the banner.

pxta diff last-month/findings.json pxta-report/findings.json

A Diff That Doesn’t Cry Wolf

A diff is only useful if two audits of an unchanged site come out the same, and that’s harder than it sounds. Client IDs and session IDs change on every visit, Google Analytics batches hits differently from one run to the next, and a crawl can reach one page more than last time.

So PXTA matches findings by what happened, not by the value that was sent: the same tracker, the same page, the same consent state, and for a leak the same kind of data through the same parameter. A page the other audit didn’t visit isn’t counted as fixed or as new, unless what happened there never happened anywhere in the old audit. A pixel that starts sending email addresses on a new sign-up page shows up. The same pixel firing on one more blog post doesn’t.

Pages are compared by path, so a staging audit can be checked against production before a release goes out.

Only New Problems Fail the Build

--fail-on medium exits with 1 only for a new medium or high finding. It also fails, at any level, for the regressions that hide everything else: a reject button PXTA can no longer press, a banner that’s gone, an audit that couldn’t load a single page.

In the GitHub Action that turns into a baseline. Keep a reviewed findings.json in the repo and let every run report what’s new since:

- uses: softdevstudio/[email protected]
  with:
    url: https://www.example.com/
    baseline: pxta-baseline.json
    baseline-fail-on: medium

Known problems stay listed in the summary but stop blocking the build. A new one fails it. When a change is accepted, the baseline file gets updated in the same pull request, so the decision is in the history.

How It Was Tested

The test suite audits a fictional bakery, then the same bakery after one planted change: the X Pixel loading on the menu page without waiting for consent. The diff has to report exactly that and nothing else. It does: two new findings, before the banner is answered and after a reject, and the pixel starting to send data on /menu in all three passes. Two audits each of the bakery and of a shop with leaking checkout pages have to come out as no change, and they do.

Three rounds of independent review went into it before release. They found staging and production audits that didn’t match, batched analytics hits that showed up as new every run, and an action that failed on problems the baseline already had. All of it was fixed and covered by tests first. The reference has the matching rules and every option, and the code is on GitHub.