EvidaLux Tools Validation Report

Every run of our 64 audit tools — behaviour lock, agreement with external references, and real-site oracles. Methodology over marketing — bad numbers publish too.
Generated: 2026-05-08T11:14:17.579377+00:00
This is an archived snapshot. For the latest, see here.

Section 1 — Cross-Tool Agreement
Diff against industry-reference tools on the same 10-site cohort. cohort: 2026-05-cohort1
1 of 14 plugins measured · median agreement 0.40
PEND rows mean the reference-tool integration is queued. Each integration may use a different per-site match metric (see plugin.metric). Median agreement is taken across implemented rows only.
PluginReference toolBandAgreementAgreed / ComparedComparison metric
accessibility.axeaxe-core CLI (Deque)PEND——Integration queued
compliance.iab_tcfIAB CMP ValidatorPEND——Integration queued
compliance.iab_tcf_verifiedIAB CMP ValidatorPEND——Integration queued
quality.lighthouse_perfGoogle PageSpeed Insights APIPEND——Integration queued
quality.owasp_zap_scanOWASP ZAP REST APIPEND——Integration queued
quality.vulnerability_nucleinuclei templates (live)PEND——Integration queued
search.lighthouse_seoGoogle PageSpeed Insights APIPEND——Integration queued
security.exposed_filesnuclei exposed-panels templatesPEND——Integration queued
security.headersMozilla Observatory v2 (MDN)RED0.404 / 10overall score within ±15 of Mozilla Observatory v2 score
security.tls_deepSSL Labs APIPEND——Integration queued
seo.broken_linkslinkchecker (W3C)PEND——Integration queued
seo.hreflang_validatorhreflang.orgPEND——Integration queued
seo.robots_txt_auditGoogle robotstxt parserPEND——Integration queued
seo.structured_dataGoogle Rich Results Test APIPEND——Integration queued

Section 2 — Real-World Correctness

Plugin findings on random real sites (US/Europe/Turkey/Arab/South America) scored against a real-browser + HTTP ground truth. Unreachable sites are INCONCLUSIVE (never gate); a gate failure is a false positive/negative on a reachable site.
RegionWhenVerdictReachableInconclusiveFPFN
arab, europe, south_america, turkey, us2026-08-04 02:51 UTCFAIL871320
arab, europe, south_america, turkey, us2026-08-03 02:36 UTCFAIL8515010
arab, europe, south_america, turkey, us2026-08-02 04:02 UTCFAIL8614010
arab, europe, south_america, turkey, us2026-08-01 03:02 UTCFAIL8317010
arab, europe, south_america, turkey, us2026-07-31 03:05 UTCFAIL8515010
arab, europe, south_america, turkey, us2026-07-30 02:46 UTCFAIL8515010
arab, europe, south_america, turkey, us2026-07-29 02:45 UTCPASS831700
arab, europe, south_america, turkey, us2026-07-28 02:52 UTCFAIL831702
arab, europe, south_america, turkey, us2026-07-27 02:39 UTCFAIL8317010
arab, europe, south_america, turkey, us2026-07-26 04:02 UTCFAIL821805
arab, europe, south_america, turkey, us2026-07-25 02:54 UTCFAIL8317010
arab, europe, south_america, turkey, us2026-07-24 02:39 UTCPASS792100
arab, europe, south_america, turkey, us2026-07-23 02:53 UTCPASS871300
arab, europe, south_america, turkey, us2026-07-22 02:38 UTCPASS851500
arab, europe, south_america, turkey, us2026-07-21 02:56 UTCPASS881200
arab, europe, south_america, turkey, us2026-07-20 02:41 UTCFAIL861410
arab, europe, south_america, turkey, us2026-07-19 02:37 UTCFAIL881210
arab, europe, south_america, turkey, us2026-07-18 20:31 UTCPASS851500
arab, europe, south_america, turkey, us2026-07-18 19:22 UTCFAIL841650
arab, europe, south_america, turkey, us2026-07-18 18:18 UTCPASS18700
arab, europe, south_america, turkey, us2026-07-18 18:00 UTCFAIL20530
arab, europe, south_america, turkey, us2026-07-18 16:09 UTCFAIL205441
arab, europe, south_america, turkey, us2026-07-04 06:08 UTCFAIL20511
arab, europe, south_america, turkey, us2026-07-03 06:20 UTCFAIL19611

Latest run — 2026-08-04 02:51 UTC

PluginConfirmedFPFNInconclusive
accessibility.eaa_mapping000156
compliance.accessibility_statement580098
compliance.age_verification00087
compliance.ai_disclosure00087
compliance.child_consent00087
compliance.cookie_consent000209
compliance.cross_border_transfer000156
compliance.dark_pattern6900156
compliance.data_subject_request160071
compliance.dpo_contact00087
compliance.eu_representative00087
compliance.geo_consistency000174
compliance.iab_tcf00087
compliance.iab_tcf_verified60081
compliance.legal_disclosure7300152
compliance.odr_link690018
compliance.pay_or_consent_wall00087
compliance.pricing_indication000174
compliance.privacy_policy_content1300166
compliance.purchase_disclosure000156
compliance.required_pages16500129
quality.api_test000217
security.exposed_files1630014
security.headers61900309
security.tls_deep2490091
seo.broken_links300025
seo.canonical_audit51104
seo.duplicate_content00089
seo.freshness370029
seo.hreflang_validator74003
seo.meta_tags157000
seo.robots_txt_audit17001
seo.sitemap84004
seo.structured_data84109
tech.dns_health34600175
tech.stack_detection00087

Wrong findings — latest run (gate)

  • false_positive seo.structured_data/schema.jsonld.present — https://ec.europa.eu/ — jsonld_blocks=0
  • false_positive seo.canonical_audit/canon.self — https://ec.europa.eu/ — canonical=https://commission.europa.eu/index_en final=https://commission.europa.eu/select-language?destination=/node/1

Past reports

Legend — column meanings

Cross-Tool Agreement
Plugin
The audit tool we are validating (e.g. seo.lighthouse_seo).
Reference tool
The industry-standard tool whose findings we compare against on the same site (e.g. PageSpeed Insights, Pa11y, W3C linkchecker, Mozilla Observatory).
Band
Bucketed from the agreement score:
GREEN agreement ≥ 0.90.
YELLOW agreement ≥ 0.70.
RED below 0.70.
Agreement
How much our plugin and the reference tool produce the same verdict, on the same site, on the same 10-site cohort (0.00–1.00). High = strong industry consensus that we are seeing the same thing.
Agreed / Compared
On each of the 10 cohort sites, did our plugin and the reference tool give the same verdict? The left number is how many sites we agreed on; the right is how many sites we could actually compare (if either tool fails on a site — timeout, blocked, 5xx — that site does not count).
Comparison metric
Short description of how the agreement number is computed for this pair (e.g. "in-domain Jaccard", "score delta ≤ 5"). The metric is plugin-specific because each reference tool exposes its findings differently.