Validation suite

Correctness is the moat. We validate in layers, and we only remove a caveat once the underlying limitation is actually fixed (never to make a screenshot look cleaner). Two layers today: Rust unit + analytical tests in the solver core, and a JS/web suite that exercises the WASM build and the Studio behaviour.

# a static server must serve the repo root (python3 -m http.server 8800)
NODE_PATH=$(npm root -g) test_suite/run_checks.sh

Where we are (2026-06)

• 288 Rust unit tests (elements, releases, plates, eigen, AS 4100/3600/4600 capacities, AISC 360, EC3, seismic, retaining wall, physical members, T/C).
• 25 closed-form analytical cases solved through the WASM build and diffed against textbook formulas.
• 91 web/Studio suites in run_checks.sh (behaviour, a11y, and live closed-form / differential / invariant checks through the browser).
• 425 catalogue sections across 6 codes (AU/US/EU/UK/CA/TH) cross-checked for internal consistency.

A consolidated, claim-by-claim record (with the test backing each claim and an honest limitations section) lives in docs/reference/validation-report.md.

Closed-form / physical-law checks

These need no values from any other program - they are equilibrium, linearity, or textbook formulas an engineer can reproduce by hand:

Check	Reference
Cantilever under UDL	fixed-end shear wL, moment wL²/2, free end 0
Simply-supported beam under UDL	reactions wL/2, midspan moment wL²/8
Cantilever / SS deflection	PL³/3EI, 5wL⁴/384EI (per-station, <0.5%)
Every demo model	ΣR + Σapplied ≈ 0 (equilibrium)
Load scaling (homogeneity)	scale all loads by k ⇒ results scale by exactly k
Superposition	response(A+B) = response(A) + response(B)
Reciprocity (Maxwell-Betti)	flexibility/stiffness is symmetric (K = Kᵀ)
Plate bending convergence	vs Timoshenko: SS 48% → 6.6% → 1.8%, clamped 99.7% → 3.7% → 0.5% at 2×2 / 4×4 / 8×8 (coarse clamped meshes lock; use ≥8 divisions/side)
Modal free vibration	cantilever 1st natural frequency vs Euler-Bernoulli f₁ = (1.875)²/(2π)·√(EI/ρAL⁴) (within 5%)
Linear buckling	pinned-pinned column critical load vs Euler Pcr = π²EI/L² (within 5%)
Tension/compression-only	X-braced panel: the compression diagonal deactivates and the shear redistributes to the tension diagonal; equilibrium holds
Section torsion / warping	solid rectangle + circle J exact (Saint-Venant); open-section J = Σbt³/3, Iw = Iy(d−tf)²/4
Section catalogue	mass = ρA, Zx = Ix/(d/2), Sx ≥ Zx, shape factor + published Ix vs closed form, across 425 sections

AS 4100 steel design (shipped, validated)

The design module is live, not aspirational. It is validated against published worked examples and tables:

Term	Validation
φMs, φMb, φNc, φNt, φVv	ASI worked examples Ex1-Ex7, within ~1%
Combined actions Cl 8.3/8.4	including the Cl 8.4.5 biaxial interaction term
Exact J / Iyy / Izz	cross-checked vs ASI catalogue (Iyy <0.5%, Izz <2.5%)
RHS / SHS / CHS capacities	vs ATM Design Capacity Tables (Grade C350/C450)

Multi-code (AISC 360 / EC3)

Beyond AS 4100, member capacity is also available to AISC 360 (US) and Eurocode 3 (EU). The cores are unit-tested in Rust, and every capacity is sanity-checked through the WASM boundary on a 360 UB - axial, major + minor flexure, lateral-torsional buckling and shear all land on first-principles values (aisc_ec3_check). This is what makes the tool usable beyond Australia.

Robustness program

The cases above are the recognisable slice. The deeper robustness work, in priority order:

1. Differential testing vs a reference solver - done for frames: random valid frames are solved in our engine and in PyNite (MIT) plus an independent numpy stiffness oracle, diffed across all 6 DOF at 0.0000% signed. Open: the same differential against an independent FE plate solver (e.g. CalculiX) on skew / aspect-ratio / point-load plates.
2. Property-based invariants - done: equilibrium (ΣR = Σapplied), linearity, superposition, symmetry, and Maxwell-Betti reciprocity all assert through the WASM solver on many models.
3. Table reconciliation - done (internal): every catalogue section is reconciled for mass-vs-area and published-Ix-vs-dimensions; this already caught and fixed real data errors. Open: an exact per-property diff against a downloaded authoritative table (ASI DCT / AISC shapes DB).
4. NAFEMS benchmarks + an independent chartered-engineer sign-off - the benchmark corpus and a consolidated validation report are in place; the named engineer sign-off is the final human gate. (open)

Differential testing earns trust in the analysis engine; the AS 4100 design trust is earned separately by the table/DCT reconciliation and the engineer sign-off. See docs/reference/validation-roadmap.md and docs/reference/validation-report.md for detail.