Independent replication · does not supersede the page it checks

Mechanism Robustness — Cross-Check

A second, independently-built pipeline over the same evidence base as Mechanism Robustness — built without reading that page's code, data or published figures. Only one comparison happens, against numbers that page's own package had already made public; every other number below was derived from scratch. Both analyses are published; neither supersedes the other.

Note, 2026-10-11. This cross-check was computed against the pre-correction version of the Mechanism Robustness page (package robustness_analysis_20260916). That page has since been rebuilt from the corrected package, which fixes the cohort pooling described below together with four other errors, so comparisons on this page refer to the earlier numbers.
Headline: a real bug was found and fixed before computing anything. The shared source table pools three non-exchangeable cohorts under one nominal cell label — for lead-slow/S0/range-clip it gives travel_time_s ≈127s (the true formal matrix), ≈97s (single verification runs) and ≈80s (an early pilot batch) for "the same" cell. After correcting for this, range_clip lands on exactly the well-known 140-run formal matrix, and the lead-slow travel-time effect reproduces to six significant figures (127.300002 → 131.440002 s, Hedges' g = 6.054). See § Data foundation.
Data foundationWhat changed before any statistic was computed

Cohort correction

Deduplication

Sanity check

Mechanism coverageHonest mapping of the plan's 4 methods onto what the data actually contains
MechanismMaps toRunsScenariosSeveritiesMetrics ≥3 sev.

Shape/geometric-noise degradation does not exist anywhere in the codebase — an exhaustive search found no injector. pilot = channel/rate exist only in lead-stopped, entirely from early pilot batches.

Trend & interaction testsFreshly implemented — no Jonckheere-Terpstra or ART ANOVA existed anywhere in this codebase before this pass

Jonckheere-Terpstra trend

ART-ANOVA interaction

Collisions (Fisher exact)

DesignMetricInteractionFp
Robustness Score0–1, trapezoidal AUC of per-severity retention · S4 total-outage excluded (see callout)
MechanismAxisMean RSn metrics
Data artifact caught: channel's S4 ("sensor destroyed") looked like perfect comfort/operation robustness — completion time falls 69.5s→15.75s, comfort metrics collapse toward zero — but this is a fast-fail/near-stop degenerate state, not genuine robustness. Excluded from every RS number here; reported as its own case study in the full report.
Breaking pointsIndependently-implemented broken-stick detector: LOOCV improvement ≥10%, slope-change CI excludes zero, bootstrap-stable location ≥70%

content_ratio × red-light-violator × travel_time

content_ratio × curved-urban-road × travel_time

The red-light breakpoint independently lands at retained ratio 0.4, close to a previously-circulated figure of 0.4567 for the same cell, with the same slope-sign pattern — resolving, in the supporting direction, a "cannot test" verdict this project's own prior skeptical audit reached on a narrower local slice.

Module attribution — mediationScoped to range_clip: the only mechanism with a no-target control scenario
OutcomeMediatorρ no-targetρ with-targetp with-target

In curved-urban-road (no target; localisation pathway isolated), localisation metrics show essentially no correlation with behaviour (all |ρ|<0.26). In target-bearing scenarios, gated recall/precision correlate strongly (ρ≈0.78, p=2.6e-13 for no-progress duration): the perception pathway, not localisation, tracks vehicle cost under range_clip.

Against the other page's published numbersThe only point this analysis reads anything from the other package — its own public README
Claim (their page)This cross-checkVerdict