SPRII

World models · Persistent representations

Shaping Persistent Representations
from Independent Interactions

Ji Dai1Quan Fang1Junyu Gao2Rongfeng Guo3Haoyan Rong4Yiping Huang1Yongxi Li2
1Beijing University of Posts and Telecommunications2Institute of Automation, Chinese Academy of Sciences3Shenzhen University4Tsinghua University

SPRII turns relations between independent interactions into supervision for persistent context. It learns what carries across encounters without numerical physical-property labels—and tests whether that information is formed, used, and useful.

Different interactions share hidden mass, drag and stiffness. Relation-guided learning creates persistent context, evaluated separately for formation, use by a fixed reader, and prediction value.
Same system. Different interactions. Changing trajectories reveal shared hidden properties. We ask what context represents, how a predictor uses it, and when it improves prediction. Schematic illustration.
Relation supervisionShape what context retains
without numerical property labels
Formation · Use · ValueSeparate what is learned,
what is used, and what helps
13 settingsPhysical systems, fields,
robotic interactions, and partners

01 / The idea

Learning from what interactions share.

An object at rest reveals little about how it will slow after a push. An earlier encounter can help, but predicting recent motion alone need not organize that information for reuse. SPRII trains on related interactions through two complementary objectives.

Align

Bring related contexts together.

Encourage agreement between the contexts of related interactions, while preserving variation across systems.

Context AContext B
Cross

Use one experience in another.

Combine a related donor’s context with the recipient’s current input to predict the recipient’s future.

Donor contextRecipient input

The relation tells us which interactions share properties; it does not provide their numerical values. Align and Cross can be studied separately.

SPRII adds relation supervision to the learner’s existing objective. History can enter through a dedicated encoder or an existing context state; each recipe uses Align, Cross, or both.

How SPRII fits JEPA, CPC, and RSSM

The Structure control provides the context route without relation objectives, isolating what Align and Cross add.

JEPA · Predict future embeddings.

In the controlled JEPA implementation, a causal Transformer encodes observation and action histories. The predictor combines this persistent context with current observation features, future actions, and the prediction horizon. SpringWorld uses Align + Cross; CoPhy JEPA uses Cross.

RSSM · Keep recurrent latent dynamics.

The CoPhy RSSM retains its deterministic and stochastic state, reconstruction, prior prediction, and KL regularization. SPRII adds a persistent-context route trained with Cross; its recurrent state remains available.

CPC · Keep future discrimination.

CoPhy CPC keeps its contrastive future-discrimination objective and native dynamics state. Cross trains its prediction route to use context from a related interaction.

Align and Cross complement the native loss. The enabled operations depend on the learner; SPRII does not impose one shared network architecture. Read the method.

02 / Three questions, separate evidence

Learned. Used. Useful.

Persistent information can be learned without being used, and used without improving prediction.

F / Formation

What does context remember?

Probe persistent physical properties in frozen representations. The relation shapes which information becomes accessible.

U / Use

Does context change the prediction?

Swap donor context while holding the reader, current input, and future actions fixed. Test whether forecasts respond to the donor’s physical properties.

V / Value

When does context help?

Compare task error with matched controls. Measure benefits across prediction horizons and readout choices.

Formation / PokeWorld

The relation selects what context retains.

When pairs share mass alone, mass is easier to read out. When they share mass and drag, drag becomes more accessible while mass becomes less accessible—even though mass is still shared.

Sharing more properties does not make every property easier to read.

Shared mass

Mass R²0.5271
Drag R²0.3075

Shared mass + drag

Mass R²0.0709
Drag R²0.9249
0Shared scale · higher is better1

Linear probes on frozen context, evaluated on validation systems. Both measured training conditions are shown.

Three donor-versus-target heatmaps for mass, drag and stiffness, with a fourth panel summarizing positive mismatched-minus-matched prediction error across three source models.
Use / SpringWorld: change the donor, keep the reader fixed. The heatmaps vary donor and target properties independently; the white diagonal marks matching values. The right panel summarizes the advantage of matching across three source models. Figure 4 in the paper · Enlarge figure.

Use / SpringWorld

Donor physics steers a fixed forecast.

Keep the fitted task reader, recipient state, and future actions fixed. Change only the donor context: forecasts have lower error for targets whose physical properties match the donor.

0.1820
Lower 16-step error for matched mass (absolute difference)

95% paired-system interval: [0.1320, 0.2343]

Mean mismatched-minus-matched error across 64 development systems, three frozen Align + Cross sources, and one fixed reader per source. Positive values favor matching.

Comparison and interpretation

Each system contributes a complete 5 × 5 donor–target grid. The difference averages 20 off-diagonal and five diagonal cells; systems and sources receive equal weight. The interval conditions on the fitted models. This is a task-reader intervention, separate from the source embedding predictor.

Full experiment in the paper

Value / SpringWorld

Keep the representation. Improve its readout.

With source features and a recipient-only base frozen, a context-conditioned residual reader reduces development MSE by 7.9% compared with a query-only residual reader. All nine source–reader pairs improve.

7.9%
Lower development MSE with the context-conditioned reader

Query-only MSE: 0.6185 · Context-conditioned MSE: 0.5695

SpringWorld Align + Cross · three sources × three readers · common targets and fitting schedules. Reader fitting changes; source features stay frozen.

Comparison and interpretation

The two routes share their architecture but not exactly their active capacity. The same reader design does not improve the tested PokeWorld families. This comparison measures readout adaptation, not a context swap through one fixed reader.

A separate control fits matched-context and mismatched-context M₂ readers with identical architecture. Matched context lowers full-development error by 8.7%, with the same ordering in all nine pairs.

Full experiment in the paper

Lower error across five primary comparisons.

Mean ± SD · lower error is better

Setting / learnerBaselineSPRIIError reduction
SpringWorld / JEPA0.138 ± 0.002
Native JEPA
0.107 ± 0.004
Align + Cross
22.3%
CoPhy Collision / CPC0.223 ± 0.006
Structure
0.189 ± 0.007
Cross
15.4%
CoPhy Collision / RSSM0.228 ± 0.005
Structure
0.202 ± 0.007
Cross
11.4%
CoPhy Collision / JEPA0.215 ± 0.007
Structure
0.184 ± 0.005
Cross
14.2%
CoPhy Balls4 / RSSM1.280 ± 0.031
Structure
1.154 ± 0.025
Cross
9.9%

The equal-weight mean error reduction across these five comparisons is 14.6%, calculated before rounding. SpringWorld uses standardized state-increment MSE; CoPhy uses physical trajectory MSE. Compare error magnitudes within each row.

Evaluation conditions and where gains depend on the task

SpringWorld evaluates 256 sealed systems using three source models and three readers per source. Its 22.3% reduction has a paired 95% confidence interval of 13.2–30.7%. CoPhy Collision uses 1,994 held-out episodes, and Balls4 uses 1,000 recipients; CoPhy results use three joint source/reader seeds, with budgets matched within each learner family.

Use experiments in SpringWorld intervene on a fixed task reader fitted to frozen representations. They do not directly test the original source predictor. The best interface also depends on the task: direct context lowers D-Clean h32 error by 22.8% relative to decoded physical parameters, but has 2.1% higher error in SpringWorld.

Project results summarize the October 2026 author manuscript linked above. The code guide identifies the available implementations and reproduction inputs.

03 / Environments

See what changes. Find what persists.

A push, a collision, a grasp, or a new partner: the visible interaction changes while a shared property carries across encounters. Explore the settings and the questions each one tests.

Controlled physical systems

Environment demoSpringWorldMass · drag · stiffness
Force and release in SpringWorld: the same spring system responds to two applied pushes. Source

Can a new interaction reuse physical context?

Persists across encounters
Mass m; Drag γ; Spring stiffness k
Changes between encounters
Initial state, actions and realized interaction history

What we learn here

Prediction MSE falls from 0.138 to 0.107 versus Native JEPA: a 22.3% reduction on 256 sealed systems (95% CI 13.2–30.7%).

Across cold-start horizons, Persistent-over-Null gains rise from 1.9% at h1 to 35.7% at h16 in the separate fixed 178-system grid.

Parameters and setup

Mass m
0.5–2 kg
Drag γ
0.25–1.5 s⁻¹
Stiffness k
4–25 N/m
Inputs, comparisons and evaluation protocol
Inputs
An independent 96-frame same-system history, the current query state and future actions.
Target
Eight-dimensional position and velocity increments over 0.8 seconds.
Comparison
SPRII Align + Cross versus Native JEPA and TDS; Structure, Align and Cross isolate the components.
Controlled conditions
The sealed comparison uses the same data bank, source/readout budgets and 256 held-out systems. Use tests freeze the representation model and fitted task reader while changing donor context.

The Use intervention acts on a task reader fitted to frozen representations; it is separate from the source embedding predictor.

Recorded dataPokeWorldMass · drag · contact stiffness
Two recorded interaction histories, replayed at quarter speed. Arrows show the actions applied to the pusher. Source

The pairing rule selects what context retains

Persists across encounters
Mass m; Drag γ; Contact stiffness k
Changes between encounters
Initial states and action modes; G1 pairs share m, G2 share (m,γ), G3 share (m,γ,k).

What we learn here

Raising correct-pair fraction from 0 to 1 increases the between/within-system context-distance ratio from 0.260 to 4.39.

Changing shared mass to shared mass + drag lowers mass-probe R² from 0.527 to 0.071 while drag-probe R² rises from 0.308 to 0.925.

Parameters and setup

Persistent factors
Mass m, drag γ and contact stiffness k
Pairing rules
G1: m; G2: m + γ; G3: m + γ + krelation settings
Factorized bank
10 × 10 × 28 physical tuplesfactor-level counts
Correct-pair fraction α
0, 0.5 or 1relation fidelity
Inputs, comparisons and evaluation protocol
Inputs
24-frame image/action histories from independent finger–object interactions, plus a current query.
Target
Future observation embeddings; physical-factor probes and task readouts assess the learned context.
Comparison
Structure, Align, Cross and Align + Cross; correct, randomized and partial-factor relations.
Controlled conditions
Relation interventions retain the source architecture, observation budget and donor/query marginals within each comparison.

The acquisition, factorized-relation and R8 fidelity banks are separate protocols; their sample counts should remain separate.

Recorded dataD-CleanDrag
A recorded D-Clean trajectory with its applied force and drag response, replayed at half speed. Source

Accurate parameter readout is not the whole prediction story

Persists across encounters
Drag γ
Changes between encounters
Initial state and force segments

What we learn here

With a common reader design, SPRII has the lowest mean error at all four horizons, including 33.7% lower h32 error than CaDM.

Direct context lowers h32 error by 22.8% relative to decoded physical parameters, despite an independently validated decoder R² above 0.996.

Parameters and setup

Persistent factor
Drag γ
Dynamics
dv/dt = F/m − γv; m = 1system definition
Integration step
Δt = 0.05simulation value; physical unit not stated
Interaction bank
Eight independent 64-step interactions per systemcontext setting
Inputs, comparisons and evaluation protocol
Inputs
A 4D state/action history and a recipient state, with an independent same-system donor.
Target
Future inertial state at horizons 1, 4, 16 and 32.
Comparison
Common-reader comparisons with TDS, NOD, FCRL, CaDM and DALI; Null, Persistent, Decode and Oracle readouts test the information interface.
Controlled conditions
Readout comparisons use frozen source features, common prediction targets and matched permitted inputs. Persistent and Decode use the same donor context.

These endpoints measure prediction through a common fitted reader; source-training budgets and method adaptations are specified separately in the paper.

Contact and object dynamics

Task schematicCoPhy CollisionObject properties · mass probes
Two abstract object interactions illustrate contact under matched physical properties, with different initial motion.
Original task schematic; not a model rollout. Source

One relation objective across three learners

Persists across encounters
Matched object properties under a scene-specific relation; mass is the reported factor-probe target
Changes between encounters
Initial conditions and independent support interactions

What we learn here

Relative to Structure, prediction error falls by 15.4% for CPC, 11.4% for RSSM and 14.2% for JEPA.

The corresponding SPRII trajectory MSEs are 0.189, 0.202 and 0.184.

Parameters and setup

Persistent factor evaluated
Object massfactor-probe target
Relation
Matched objects under a scene-specific rulerelation setting
Context / target
3 supports + 3 query frames → 12 future framestask setting
Inputs, comparisons and evaluation protocol
Inputs
Three independent support histories and the first three query frames.
Target
The next 12 physical trajectory frames.
Comparison
SPRII Cross versus Native, Structure and Random across CPC, RSSM and JEPA.
Controlled conditions
Each learner comparison shares queries, targets, donor eligibility and training budgets across arms; the main result uses 1,994 held-out episodes and three joint source/reader seeds.

The multi-support prediction protocol differs from the original CoPhy task’s input and target intervals.

Task schematicCoPhy BallsMass · friction · restitution
Two abstract four-ball arrangements illustrate rebound behavior under related object physics.
Original task schematic; not a model rollout. Source

Persistent context in another contact system

Persists across encounters
Mass; Friction; Restitution; the principal reported probe target
Changes between encounters
Initial conditions and independent support interactions

What we learn here

On held-out Balls4 recipients, RSSM MSE falls from 1.280 to 1.154 versus Structure, a 9.9% reduction.

Restitution dominates the reported reader-memory probes and fixed-predictor donor effects; mass and friction are substantially weaker.

Parameters and setup

Physical factors tested
Mass, friction and restitution
Context / target
3 supports + 3 query frames → 27 future framestask setting
Inputs, comparisons and evaluation protocol
Inputs
Three independent support histories and the first three query frames.
Target
The next 27 ball-trajectory frames.
Comparison
Native, Structure and Random where tested; Cross for JEPA/CPC/RSSM and Align + Cross for CoPhyNet.
Controlled conditions
The held-out RSSM comparison uses the same 1,000 Balls4 recipients. Factor-specific donor tests keep source/reader weights and recipient inputs fixed.

Balls4 is a task configuration within Balls. Development reader-memory assays and the held-out endpoint use different stated populations.

Task schematicCoPhy BlocktowerObject properties · gravity probes
Two abstract block towers illustrate different contacts and motion under shared vertical gravity.
Original task schematic; not a model rollout. Source

Where relation supervision fails to help

Persists across encounters
Matched object properties under a scene-specific relation; vertical gravity is the reported factor-probe target
Changes between encounters
Initial conditions and independent support interactions.

What we learn here

Blocktower is one of three scenes in the 45-cell CoPhy development matrix spanning four learner families.

The additional JEPA sources retain an adverse contrast: Cross has higher prediction error than Random at both reported sources.

Parameters and setup

Persistent factor evaluated
Vertical gravityfactor-probe target
Relation
Matched objects under a scene-specific rulerelation setting
Context / target
3 supports + 3 query frames → 27 future framestask setting
Inputs, comparisons and evaluation protocol
Inputs
Three support histories and the first three query frames, through the fixed cached perception route.
Target
The next 27 frames of multibody motion.
Comparison
JEPA, CPC, RSSM and CoPhyNet across available Native, Structure and Random controls.
Controlled conditions
Perception, query/target construction and pose-head route are held fixed within the scene.

These are development comparisons, including reader-selection examples; gains depend on learner and comparator.

Evolving fields

Recorded dataBurgersViscosity
One released Burgers trajectory, shown as a spatial profile and a space–time field. Source

Shared viscosity across different fields

Persists across encounters
Viscosity ν
Changes between encounters
Initial spatial field

What we learn here

SPRII lowers mean error relative to released NOD on all three Burgers distributions.

Against matched prediction-only continuation, the ID gain is 4.1%, while OOD performance is near-tied.

Parameters and setup

Persistent factor
Viscosity ν
Illustrated trajectory
ν = 0.01illustration-specific value; not a range
Illustrated resolution
401 spatial samples and 101 saved timesdisplayed trajectory setting
Inputs, comparisons and evaluation protocol
Inputs
An independent same-viscosity trajectory and the recipient’s spatial field.
Target
Future 1D Burgers fields.
Comparison
Released NOD, Random and prediction-only continued NOD; related operator adaptations are reported separately.
Controlled conditions
Continuation controls match additional update budgets, with the same prediction task and distribution-specific evaluations.

The released-default and continued-NOD comparisons answer different questions; the illustrated ν value is not the dataset range.

Environment demoFitzHugh–NagumoReaction parameters
FitzHugh–Nagumo field evolution from a numerical solver, showing the coupled u and v fields. Source

Reaction parameters across different initial fields

Persists across encounters
Reaction parameters k and β
Changes between encounters
Independent Gaussian-random-field initial conditions

What we learn here

Increasing history count from one to four reduces SPRII H50 error by 16.9%, 18.0% and 10.9% on ID, OOD-Intra and OOD-Extra.

The gap to NOD narrows on every split, although NOD retains slightly lower absolute mean error at four histories.

Parameters and setup

Persistent factors
Reaction parameters k and β
Grid
Periodic 128 × 128spatial setting
Integration / saving
RK4 with Δt = 0.001; 101 frames through t = 10simulation setting; physical time unit not stated
Illustrated trajectory
k = 0.03, β = 0.20illustration-specific values; not ranges
Inputs, comparisons and evaluation protocol
Inputs
One, two or four related field histories; each uses frames 0, 25, 50, 75 and 100.
Target
Evolution of coupled u and v fields, including H50 prediction.
Comparison
NOD-Hier versus the SPRII alignment/continuation recipe on a regenerated Python simulation bank.
Controlled conditions
Each recipe is held fixed as history count changes; the reporting initial conditions and three source seeds are shared.

The recipes use different source-training budgets: 45,000 updates for NOD and 50,000 for SPRII. This comparison does not isolate alignment at equal compute.

Recorded robotic interactions

Recorded dataBaxter tactileHardness · shape
A recorded grasp, viewed through 16 tactile sensors. The replay reveals the stored samples in order. Source

The relation selects hardness or shape

Persists across encounters
Object hardness category; Object shape: cube or cylinder
Changes between encounters
Independent grasps; same-hardness pairs change shape, same-shape pairs change hardness

What we learn here

Hardness accuracy is 1.000 under same-hardness relations versus 0.333 under same-shape relations.

Shape balanced accuracy reverses the ordering: 1.000 under same-shape relations versus 0.500 under same-hardness relations.

Parameters and setup

Shape categories
Cube and cylinderobject categories
Hardness categories
Three shared levels, indexed 0–2category labels; not physical hardness units
Crossed bank
6 configurations × 170 grasps; 110/30/30 train/validation/confirmation grasps per configurationdata setting
Inputs, comparisons and evaluation protocol
Inputs
40-step causal tactile histories drawn from 80 × 16 grasp-peak arrays.
Target
Future tactile representations, with hardness and shape readouts.
Comparison
Same-hardness, same-shape and randomized relations, with Align + Cross.
Controlled conditions
Each batch balances configurations and donor/query marginals; confirmation holds out grasps from the same six known configurations.

The evidence concerns held-out grasps from known object configurations. Available recordings do not include pre-contact frames or action sequences.

Environment demoRH20TTask context
An official RH20T drawer-opening demonstration, separate from the episodes used in our evaluation. Source

Can another episode inform force and motion forecasts?

Persists across encounters
Task identity across different recorded episodes
Changes between encounters
Recorded episode; current visual, robot-state and action inputs

What we learn here

Under matched donors, Cross lowers error on all five reported force/torque and TCP endpoints relative to Structure.

Same-task independent episodes provide the persistent context; known physical-parameter equivalence is not assumed.

Parameters and setup

Persistent relation
Task identitycontext category
Main forecasting population
25 held-out tasksevaluation setting
Hard-negative bank
24 tasks, analyzed separatelyevaluation setting
Inputs, comparisons and evaluation protocol
Inputs
Synchronized visual, robot-state and action histories from different episodes of the same task.
Target
Normalized 6D force/torque and 3D TCP motion.
Comparison
Cross and Align + Cross versus Structure, donor-pairing and input controls.
Controlled conditions
Within donor interventions, the query, target, horizon, actions and fitted predictor stay fixed while context changes.

These are offline forecasting and donor-use results from recorded manipulation, not a newly executed closed-loop robot controller.

Control and recurring partners

Environment demoSwimmerLink masses · joint damping
A three-link swimmer starts at rest, drives its joints, reverses the drive, and then turns the motors off. Source

The right context improves a frozen predictor

Persists across encounters
Three link masses; Two joint damping parameters
Changes between encounters
State/action interactions; own-system versus wrong-system persistent update

What we learn here

Own-system updates reduce error for all three frozen predictors on 512 prospective systems.

For Persistent-JEPA, wrong-minus-own MSE is 5.329 × 10⁻⁵ (paired 95% CI 3.930–6.724 × 10⁻⁵).

Parameters and setup

Persistent physical factors
3 link masses and 2 joint damping parameters
Prospective population
512 new systemsevaluation setting
Inputs, comparisons and evaluation protocol
Inputs
A persistent update plus the recipient’s current state/action query.
Target
State responses to actions in a three-link MuJoCo swimmer.
Comparison
Own-system versus wrong-system updates within each of three frozen JEPA/GRU predictors.
Controlled conditions
Architecture, predictor weights and training-only normalization remain fixed within each comparison.

This is a fixed-update intervention study, without new Align/Cross source training.

Environment demoOvercookedPartner behavior
Two scripted cooks coordinate food preparation and delivery in the Overcooked environment. Source

Does recognizing a partner improve cooperation?

Persists across encounters
Fixed neural partner behavior / identity
Changes between encounters
Locations, actions and outcomes across disjoint episodes

What we learn here

Same-partner alignment lowers partner-behavior probe error at every history budget. At full history it improves familiar-partner return in all three seeds, while gains with held-out partners remain inconsistent.

A more informative context does not guarantee better cooperation.

Parameters and setup

Persistent relation
Partner identity / behaviorcontext category
Layout
grounded_coord_simpleenvironment setting
Partners
20 familiar partners; 2 held-out development partnersevaluation setting
History budgets
32, 64 or 128 histories per familiar partnercontext setting
Inputs, comparisons and evaluation protocol
Inputs
Disjoint prior episodes with the same neural partner and the current action query.
Target
Ego-action prediction, partner-behavior readout and cooperation return.
Comparison
Align-only history conditioning versus a matched history slot with variance/covariance regularization.
Controlled conditions
Matched conditions share architecture, initialization and training setup; prior episodes identify the partner without exposing policy weights.

The study uses one layout; familiar-partner return and the held-out development cohort are reported separately.

Environment demoPendulumMass · length
A fixed sequence of torques drives the pendulum; the display follows the resulting angle changes. Source

Dynamics outside the training range

Persists across encounters
Mass; Length / system dynamics
Changes between encounters
Episodes, initial states and control interactions; mass/length vary between systems

What we learn here

SPRII has higher mean return than CaDM in four of five groups, with lower return in OOD c2.

Matched and wrong-context tests use the same checkpoint; the paper reports return, success and physical probes together.

Parameters and setup

Persistent factors
Mass and length
Evaluation groups
1 ID + 4 OOD groupsevaluation setting
Rollouts
10 episodes × 200 steps per group and training seedcontrol setting
Upright success
Final 100 post-action states all within 60° of uprightmetric definition
Inputs, comparisons and evaluation protocol
Inputs
Transition history and the current pendulum state through the CaDM interface.
Target
Control return and upright success, paired with physical probes and fixed-model donor tests.
Comparison
One ID-selected SPRII recipe versus CaDM and Random relation, evaluated across all groups.
Controlled conditions
The same ID-selected recipe and success criterion apply to every OOD group; there is no per-group recipe selection.

Weak mass probes do not establish recovery of every physical parameter. Logged-action donor interventions are distinct from online controller interventions.

04 / Build on this work

Run SPRII.

The official public repository includes source code, environment adapters, experiment recipes, and a small CPU example to check the installation.

The code map connects each question and setting to its implementation.

  1. Start with the quick example.Install the core dependencies and run the documented small CPU workflow.
  2. Choose an experiment recipe.Follow the environment-specific data setup and training or evaluation instructions.
  3. Match the release to the result.Use the release notes and recipe map to identify supported experiments and required artifacts.
What you can run. Start with the documented CPU example, then select a setting-specific recipe. The repository includes source, configurations and tests. External datasets and dependencies have setup instructions; trained checkpoints and historical run logs are not bundled. Reproduction guide.

Citation

@misc{dai2026sprii,
  title   = {Shaping Persistent Representations from Independent Interactions},
  author  = {Dai, Ji and Fang, Quan and Gao, Junyu and Guo, Rongfeng
             and Rong, Haoyan and Huang, Yiping and Li, Yongxi},
  note    = {Author manuscript, October 2026},
  year    = {2026},
  url     = {https://jidaivita.github.io/sprii/SPRII.pdf}
}
Media sources and credits

Environment demonstrations and task schematics show the setting; the reported findings above come from the paper experiments.

  • SpringWorld. Project simulation and visualization; MuJoCo contributors. Two-body colors follow the paper renderer: object 1 is deep teal, object 2 is orange. All 201 archived states and their actions retain their original order, timing, geometry and camera; only the two display materials change. Playback follows simulation time, sampled every 0.05 s. Source.
  • PokeWorld. Project recordings, shown with the same orange object, gray pusher and light floor as the paper. All 24 stored states remain in source order, without interpolation or a new simulation. The viewing area is wider for the video so both complete trajectories stay visible. Action arrows show normalized inputs; the recorded disk geometry determines contact. This color visualization is separate from the model’s 64×64 observation renderer.
  • D-Clean. Project numerical trajectory and visualization. Preserves all 64 stored state and force samples. Black arrows show applied force; gray dashed arrows show −mγv. The final unavailable outgoing force remains unavailable rather than being replaced with zero. Playback is half physical speed.
  • Burgers. Burgers data: Zituo Chen · CC BY 4.0 · redrawn with RdBu_r. All 101 released numerical states retain their values and order. They span simulation time 0–5 s and are presented at 10 frames/s. Poster selects frame 50. Source trajectory SHA-256: 57f6aff9b412f3cd597f7c9d76ae300abc8a16a8607c6c85d07d9bc502a780dd. Source. License.
  • FitzHugh–Nagumo. Based on Zituo Chen’s DR2D generator · Python solver illustration. Existing Python reimplementation of the released equations, periodic stencil and RK4 integration, seed 42, k = 0.03, β = 0.20. Uses the upstream summer palette and fixed shared signed limits. Contains 101 states spanning 0–10 s at 10 frames/s; final-state poster. The separate software-record license is not inferred from the Burgers dataset license. Source. License.
  • Baxter tactile. Data: Youssef Amine, Christian Gianoglio and Maurizio Valle · CC BY 4.0. All 80 stored samples of the selected 16-channel grasp are retained, without smoothing or fabricated values. Playback reveals 10 samples per display second and holds the final state for one second; this is a display rate, not an acquisition-rate claim. Source. License.
  • RH20T. Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Chenxi Wang, Junbo Wang, Haoyi Zhu and Cewu Lu / RH20T. Demo: Hao-Shu Fang et al. / RH20T · CC BY-SA 4.0 · cropped and resized. Preserves all 260 consecutive frames and the original 30 frames/s presentation rate; no interpolation, speed changes or invented motion. Poster is source frame 225. Source original SHA-256: c0a02599ced9c907728d2d8dac8c7858fdee3f39835fbd74a4a137c63c29b8d1. Source. Reference. Reference. License.
  • Swimmer. Project simulation and visualization; MuJoCo contributors. The light floor and deep-teal body match the paper. The original three-link geometry, recorded trajectory and opposing joint torques are unchanged: rest (0–1 s), drive (1–4 s), reversal (4–7 s), then motors off (7–10 s). Every fourth stored state is shown at 25 frames/s, without interpolation. Grid, gray trail and torque labels are display overlays. Source.
  • Overcooked. Rendering adapted from JaxMARL contributors (Alexander Rutherford et al.). The archived 120-action sequence is replayed with the original environment and seed; all recorded positions, inventories, recipes and rewards match, and four original-renderer frames match the archived images pixel for pixel. Only display colors change: warm-white floor, pale counters, purple/gold partners and gray pots. All 121 frames retain their order and 120 ms display duration. The paper thumbnail uses the same rendered board. This is a UtilityGreedy controller demonstration, not a trained-policy result. Source. Reference. Reference.
  • Pendulum. CaDM: Kimin Lee, Younggyo Seo, Seunghyun Lee, Honglak Lee and Jinwoo Shin; Gym: OpenAI. Uses unchanged CaDM reset/step and normalization methods with a fixed input and seed 20260926. The original Gym geometry and colors were previously redrawn with Pillow; these are not native Gym pixels. Contains 161 real simulated states at 20 frames/s, with torque and response overlays. This curation only copies the pre-existing output. OpenAI Gym 0.16: MIT. CaDM: no top-level license identified in the existing source audit. Source. Reference. Reference.
  • CoPhy Collision. Original task schematic by the SPRII project, drawn from simple vector geometry. Arrows are explanatory motion cues. CoPhy benchmark.
  • CoPhy Balls. Original task schematic by the SPRII project, drawn from simple vector geometry. Arrows are explanatory motion cues. CoPhy benchmark.
  • CoPhy Blocktower. Original task schematic by the SPRII project, drawn from simple vector geometry. Arrows are explanatory motion cues. CoPhy benchmark.

Overview figure and original task schematics by the SPRII authors. Code licenses and third-party notices. Contact: Ji Dai.