Verification/Methodology
Free Lab
METHOD

How verification
works.

Layered gates on measured behavior. Bounded claims, published uncertainty, missing evidence first.

SYNTHETIC REPLAY / SAME SCENE FAMILY AS THE FAILURE LAB

Measure the failure mode

Trace probes where possible; explicitly toy callables where required.

SYNTHETIC
Same-frame fixture traces; draft boundaries.
01 / FAILURE MODES

Five ways physics
breaks the demo.

Every gate traces to a named failure mode with a measurement, never a vibe.

FM1

Compounding autoregressive drift

Latent errors accumulate over horizon T until the world model leaves the real state space: interpenetration, boundary breach, phantom contact.

E[T] ~ O(T·ε)
FM2

The executability gap

Perceptual similarity (LPIPS, SSIM) says the motion looks right while commanded forces exceed torque, contact, or acceleration limits.

F_contact = μ·F_N
FM3

Perceptual hallucination

Out-of-distribution inputs collapse to the nearest latent codebook vector. The reconstruction residual trips the gate before control executes.

ALERT IF E_recon > τ
FM4

Action marginalization

The model plays back plausible motion while ignoring the action stream. Shuffled actions yielding identical predictions proves it.

SHUFFLE TEST ≈ 0
FM5

Sim-to-real degradation

Positional divergence between simulated and physical trajectories in identical reference frames, measured as discrete Fréchet distance.

FRÉCHET(P_sim, P_real)

Match the model to the measurement

An architecture name does not establish causal understanding or physical validity. Record the model, task, observation and action contract.

Three schools

ARCHITECTURE / CAUSAL RUNG
SCHOOLRUNG
RENDERER SCHOOLPixel-space video modelsVisual plausibility is not contact-dynamics evidence. Verify timing, action conditioning and physical constraints for the specific model and task.PIXELS
LATENT / JEPA SCHOOLLatent prediction and planningLatent rollouts can support planning. Inference speed, uncertainty and safety properties need model-specific measurements.LATENT STATE
SPATIAL / ENGINEERED SCHOOLVoxels + PINN constraintsMetric geometry with conservation-law penalties. The verifiable structure behind industrial digital twins and audits.RUNG II
03 / SYNTHETIC GOVERNANCE

Accumulate,
then filter.

Recursive training on uncurated synthetic data collapses variance and erases the tail events where safety lives. The fix is a law, not a hope.

D(n+1) = α·D_real + (1−α)·{x ∈ G : Verifier(x)}REAL RATIO α PRESERVED / EVERY SYNTHETIC SAMPLE VERIFIED

Declare the data mixture

Real fraction alpha is separate from the synthetic-fraction gate.

ILLUSTRATION
Real 50%Synthetic 50%GOVERNED

This ratio illustration is not a prediction of model collapse. A retained real anchor, verifier quality and labeled validation remain necessary.

Run the provenance checks
04 / ENGINE MAP

Seven modules.
One chain of custody.

Local-first and air-gap-compatible: one service, one data directory, complete backup by archiving the folder.

Module responsibilities

M1–M7 / TYPED CORE
MODULEOUTPUT
M1Degradation ScorerSim-to-real divergence, horizon drift, action sensitivity → scorecard JSON.CORE
M2Executability GuardAction-chunk physics validation → supervisory HALT within one chunk of latency.CORE
M3Bounded VerifierExact queries over sub-policies → PROVEN / VIOLATED / UNKNOWN certificates.PROOF
M4Adversarial EngineSeeded friction / mass / noise search → edge-case catalogue with replay refs.HUNT
M5Embodiment BenchmarkCross-robot transfer with frame checks → centered-Fréchet transfer score.BENCH
M6Provenance LedgerRun seals, dataset cards, reviews → append-only SHA-256 hash chain.CHAIN
M7Compliance MappingScorecards + ledger heads → regulation-mapped dossier packs + OTA vault.FILE

Reproducibility, with a recorded scope

Inputs, profile, code version and seed are pinned. Same-platform determinism is tested; verify a second platform with the supplied golden-check tool.

Canonical inputs

Schema validation, key sorting, unit conversion, and reference-frame pinning before anything scores.

Double precision

Measurements use unrounded floating-point values. Display rounding does not change gate decisions; cross-platform reproduction requires the golden check.

Seeded stochasticity

Every random operation takes an explicit, recorded seed. Same seed, same scorecard.

Timestamp-free hashes

Canonical bodies exclude wall-clock time, so independent re-runs on the same platform hash byte-identically.

TASK-SPACE TRANSFER ONLY / JOINT-SPACE CROSS-EMBODIMENT REFUSED LOUDLY / RATE RESAMPLING OPEN (M5b)
DRAFT = UNCALIBRATED / PROXY = APPROXIMATION / SYNTHETIC = GENERATED / TAMPER-EVIDENT = HASH-CHAINED

Verify the method
on your hardest case.

ONE ROBOT / ONE FAILURE MODE