The idea
Capture a short physical event, then use an offline agent to propose a model, generate a small runnable simulation, fit its parameters against the recording, and compare the rendered prediction with the real observation. Processing may take hours.
How it works
- Record synchronized reusable capture.trix evidence of an event, including available video, depth, camera pose, audio, tracks, collisions, scale, motion, and timing and calibration metadata.
- An offline agent proposes a model, generates a small runnable simulation (for example, in Rust), sweeps parameters such as friction, mass, bounce, gravity, and ramp angle, then renders predictions against the captured event.
- If residuals persist, revise the model—for example, replace a rigid hinge with a spring. First reproduce the observed event; then edit the model and compare a counterfactual prediction with a new real capture.
AI’s role: Propose editable model structures and generate small runnable simulation code; fitting must compare simulated positions and collision timing with recorded observations, while ambiguous or hidden forces remain explicit.
First demonstration
Measure a ramp and marble, record a short roll and bounce, then fit a simple model to its trajectory and collision timing. Change the ramp angle or friction, predict the result, and compare it with a new recording of the marble.
What to solve next
Can the system distinguish competing physical models when a short recording leaves mass, friction, hidden forces, or collision details ambiguous or non-identifiable?