SPRII turns relations between independent interactions into supervision for persistent context. It learns what carries across encounters without numerical physical-property labels—and tests whether that information is formed, used, and useful.
Same system. Different interactions. Changing trajectories reveal shared hidden properties. We ask what context represents, how a predictor uses it, and when it improves prediction. Schematic illustration.
Relation supervisionShape what context retains without numerical property labels
Formation · Use · ValueSeparate what is learned, what is used, and what helps
13 settingsPhysical systems, fields, robotic interactions, and partners
01 / The idea
Learning from what interactions share.
An object at rest reveals little about how it will slow after a push. An earlier encounter can help, but predicting recent motion alone need not organize that information for reuse. SPRII trains on related interactions through two complementary objectives.
Align
Bring related contexts together.
Encourage agreement between the contexts of related interactions, while preserving variation across systems.
Context A≈Context B
Cross
Use one experience in another.
Combine a related donor’s context with the recipient’s current input to predict the recipient’s future.
Donor context+Recipient input
The relation tells us which interactions share properties; it does not provide their numerical values. Align and Cross can be studied separately.
SPRII adds relation supervision to the learner’s existing objective. History can enter through a dedicated encoder or an existing context state; each recipe uses Align, Cross, or both.
How SPRII fits JEPA, CPC, and RSSM
The Structure control provides the context route without relation objectives, isolating what Align and Cross add.
JEPA · Predict future embeddings.
In the controlled JEPA implementation, a causal Transformer encodes observation and action histories. The predictor combines this persistent context with current observation features, future actions, and the prediction horizon. SpringWorld uses Align + Cross; CoPhy JEPA uses Cross.
RSSM · Keep recurrent latent dynamics.
The CoPhy RSSM retains its deterministic and stochastic state, reconstruction, prior prediction, and KL regularization. SPRII adds a persistent-context route trained with Cross; its recurrent state remains available.
CPC · Keep future discrimination.
CoPhy CPC keeps its contrastive future-discrimination objective and native dynamics state. Cross trains its prediction route to use context from a related interaction.
Align and Cross complement the native loss. The enabled operations depend on the learner; SPRII does not impose one shared network architecture. Read the method.
02 / Three questions, separate evidence
Learned. Used. Useful.
Persistent information can be learned without being used, and used without improving prediction.
F / Formation
What does context remember?
Probe persistent physical properties in frozen representations. The relation shapes which information becomes accessible.
U / Use
Does context change the prediction?
Swap donor context while holding the reader, current input, and future actions fixed. Test whether forecasts respond to the donor’s physical properties.
V / Value
When does context help?
Compare task error with matched controls. Measure benefits across prediction horizons and readout choices.
Formation / PokeWorld
The relation selects what context retains.
When pairs share mass alone, mass is easier to read out. When they share mass and drag, drag becomes more accessible while mass becomes less accessible—even though mass is still shared.
Sharing more properties does not make every property easier to read.
Shared mass
Mass R²0.5271
Drag R²0.3075
Shared mass + drag
Mass R²0.0709
Drag R²0.9249
0Shared scale · higher is better1
Linear probes on frozen context, evaluated on validation systems. Both measured training conditions are shown.
Use / SpringWorld: change the donor, keep the reader fixed. The heatmaps vary donor and target properties independently; the white diagonal marks matching values. The right panel summarizes the advantage of matching across three source models. Figure 4 in the paper · Enlarge figure.
Use / SpringWorld
Donor physics steers a fixed forecast.
Keep the fitted task reader, recipient state, and future actions fixed. Change only the donor context: forecasts have lower error for targets whose physical properties match the donor.
0.1820
Lower 16-step error for matched mass (absolute difference)
95% paired-system interval: [0.1320, 0.2343]
Mean mismatched-minus-matched error across 64 development systems, three frozen Align + Cross sources, and one fixed reader per source. Positive values favor matching.
Comparison and interpretation
Each system contributes a complete 5 × 5 donor–target grid. The difference averages 20 off-diagonal and five diagonal cells; systems and sources receive equal weight. The interval conditions on the fitted models. This is a task-reader intervention, separate from the source embedding predictor.
With source features and a recipient-only base frozen, a context-conditioned residual reader reduces development MSE by 7.9% compared with a query-only residual reader. All nine source–reader pairs improve.
7.9%
Lower development MSE with the context-conditioned reader
SpringWorld Align + Cross · three sources × three readers · common targets and fitting schedules. Reader fitting changes; source features stay frozen.
Comparison and interpretation
The two routes share their architecture but not exactly their active capacity. The same reader design does not improve the tested PokeWorld families. This comparison measures readout adaptation, not a context swap through one fixed reader.
A separate control fits matched-context and mismatched-context M₂ readers with identical architecture. Matched context lowers full-development error by 8.7%, with the same ordering in all nine pairs.
The equal-weight mean error reduction across these five comparisons is 14.6%, calculated before rounding. SpringWorld uses standardized state-increment MSE; CoPhy uses physical trajectory MSE. Compare error magnitudes within each row.
Evaluation conditions and where gains depend on the task
SpringWorld evaluates 256 sealed systems using three source models and three readers per source. Its 22.3% reduction has a paired 95% confidence interval of 13.2–30.7%. CoPhy Collision uses 1,994 held-out episodes, and Balls4 uses 1,000 recipients; CoPhy results use three joint source/reader seeds, with budgets matched within each learner family.
Use experiments in SpringWorld intervene on a fixed task reader fitted to frozen representations. They do not directly test the original source predictor. The best interface also depends on the task: direct context lowers D-Clean h32 error by 22.8% relative to decoded physical parameters, but has 2.1% higher error in SpringWorld.
Project results summarize the October 2026 author manuscript linked above. The code guide identifies the available implementations and reproduction inputs.
03 / Environments
See what changes. Find what persists.
A push, a collision, a grasp, or a new partner: the visible interaction changes while a shared property carries across encounters. Explore the settings and the questions each one tests.
Force and release in SpringWorld: the same spring system responds to two applied pushes. Source
Can a new interaction reuse physical context?
Persists across encounters
Mass m; Drag γ; Spring stiffness k
Changes between encounters
Initial state, actions and realized interaction history
What we learn here
Prediction MSE falls from 0.138 to 0.107 versus Native JEPA: a 22.3% reduction on 256 sealed systems (95% CI 13.2–30.7%).
Across cold-start horizons, Persistent-over-Null gains rise from 1.9% at h1 to 35.7% at h16 in the separate fixed 178-system grid.
Parameters and setup
Mass m
0.5–2 kg
Drag γ
0.25–1.5 s⁻¹
Stiffness k
4–25 N/m
Inputs, comparisons and evaluation protocol
Inputs
An independent 96-frame same-system history, the current query state and future actions.
Target
Eight-dimensional position and velocity increments over 0.8 seconds.
Comparison
SPRII Align + Cross versus Native JEPA and TDS; Structure, Align and Cross isolate the components.
Controlled conditions
The sealed comparison uses the same data bank, source/readout budgets and 256 held-out systems. Use tests freeze the representation model and fitted task reader while changing donor context.
The Use intervention acts on a task reader fitted to frozen representations; it is separate from the source embedding predictor.
A recorded D-Clean trajectory with its applied force and drag response, replayed at half speed. Source
Accurate parameter readout is not the whole prediction story
Persists across encounters
Drag γ
Changes between encounters
Initial state and force segments
What we learn here
With a common reader design, SPRII has the lowest mean error at all four horizons, including 33.7% lower h32 error than CaDM.
Direct context lowers h32 error by 22.8% relative to decoded physical parameters, despite an independently validated decoder R² above 0.996.
Parameters and setup
Persistent factor
Drag γ
Dynamics
dv/dt = F/m − γv; m = 1system definition
Integration step
Δt = 0.05simulation value; physical unit not stated
Interaction bank
Eight independent 64-step interactions per systemcontext setting
Inputs, comparisons and evaluation protocol
Inputs
A 4D state/action history and a recipient state, with an independent same-system donor.
Target
Future inertial state at horizons 1, 4, 16 and 32.
Comparison
Common-reader comparisons with TDS, NOD, FCRL, CaDM and DALI; Null, Persistent, Decode and Oracle readouts test the information interface.
Controlled conditions
Readout comparisons use frozen source features, common prediction targets and matched permitted inputs. Persistent and Decode use the same donor context.
These endpoints measure prediction through a common fitted reader; source-training budgets and method adaptations are specified separately in the paper.
Three independent support histories and the first three query frames.
Target
The next 12 physical trajectory frames.
Comparison
SPRII Cross versus Native, Structure and Random across CPC, RSSM and JEPA.
Controlled conditions
Each learner comparison shares queries, targets, donor eligibility and training budgets across arms; the main result uses 1,994 held-out episodes and three joint source/reader seeds.
The multi-support prediction protocol differs from the original CoPhy task’s input and target intervals.
Increasing history count from one to four reduces SPRII H50 error by 16.9%, 18.0% and 10.9% on ID, OOD-Intra and OOD-Extra.
The gap to NOD narrows on every split, although NOD retains slightly lower absolute mean error at four histories.
Parameters and setup
Persistent factors
Reaction parameters k and β
Grid
Periodic 128 × 128spatial setting
Integration / saving
RK4 with Δt = 0.001; 101 frames through t = 10simulation setting; physical time unit not stated
Illustrated trajectory
k = 0.03, β = 0.20illustration-specific values; not ranges
Inputs, comparisons and evaluation protocol
Inputs
One, two or four related field histories; each uses frames 0, 25, 50, 75 and 100.
Target
Evolution of coupled u and v fields, including H50 prediction.
Comparison
NOD-Hier versus the SPRII alignment/continuation recipe on a regenerated Python simulation bank.
Controlled conditions
Each recipe is held fixed as history count changes; the reporting initial conditions and three source seeds are shared.
The recipes use different source-training budgets: 45,000 updates for NOD and 50,000 for SPRII. This comparison does not isolate alignment at equal compute.
Two scripted cooks coordinate food preparation and delivery in the Overcooked environment. Source
Does recognizing a partner improve cooperation?
Persists across encounters
Fixed neural partner behavior / identity
Changes between encounters
Locations, actions and outcomes across disjoint episodes
What we learn here
Same-partner alignment lowers partner-behavior probe error at every history budget. At full history it improves familiar-partner return in all three seeds, while gains with held-out partners remain inconsistent.
A more informative context does not guarantee better cooperation.
Parameters and setup
Persistent relation
Partner identity / behaviorcontext category
Layout
grounded_coord_simpleenvironment setting
Partners
20 familiar partners; 2 held-out development partnersevaluation setting
History budgets
32, 64 or 128 histories per familiar partnercontext setting
Inputs, comparisons and evaluation protocol
Inputs
Disjoint prior episodes with the same neural partner and the current action query.
Target
Ego-action prediction, partner-behavior readout and cooperation return.
Comparison
Align-only history conditioning versus a matched history slot with variance/covariance regularization.
Controlled conditions
Matched conditions share architecture, initialization and training setup; prior episodes identify the partner without exposing policy weights.
The study uses one layout; familiar-partner return and the held-out development cohort are reported separately.
A fixed sequence of torques drives the pendulum; the display follows the resulting angle changes. Source
Dynamics outside the training range
Persists across encounters
Mass; Length / system dynamics
Changes between encounters
Episodes, initial states and control interactions; mass/length vary between systems
What we learn here
SPRII has higher mean return than CaDM in four of five groups, with lower return in OOD c2.
Matched and wrong-context tests use the same checkpoint; the paper reports return, success and physical probes together.
Parameters and setup
Persistent factors
Mass and length
Evaluation groups
1 ID + 4 OOD groupsevaluation setting
Rollouts
10 episodes × 200 steps per group and training seedcontrol setting
Upright success
Final 100 post-action states all within 60° of uprightmetric definition
Inputs, comparisons and evaluation protocol
Inputs
Transition history and the current pendulum state through the CaDM interface.
Target
Control return and upright success, paired with physical probes and fixed-model donor tests.
Comparison
One ID-selected SPRII recipe versus CaDM and Random relation, evaluated across all groups.
Controlled conditions
The same ID-selected recipe and success criterion apply to every OOD group; there is no per-group recipe selection.
Weak mass probes do not establish recovery of every physical parameter. Logged-action donor interventions are distinct from online controller interventions.
The code map connects each question and setting to its implementation.
Start with the quick example.Install the core dependencies and run the documented small CPU workflow.
Choose an experiment recipe.Follow the environment-specific data setup and training or evaluation instructions.
Match the release to the result.Use the release notes and recipe map to identify supported experiments and required artifacts.
What you can run. Start with the documented CPU example, then select a setting-specific recipe. The repository includes source, configurations and tests. External datasets and dependencies have setup instructions; trained checkpoints and historical run logs are not bundled. Reproduction guide.
Citation
@misc{dai2026sprii,
title = {Shaping Persistent Representations from Independent Interactions},
author = {Dai, Ji and Fang, Quan and Gao, Junyu and Guo, Rongfeng
and Rong, Haoyan and Huang, Yiping and Li, Yongxi},
note = {Author manuscript, October 2026},
year = {2026},
url = {https://jidaivita.github.io/sprii/SPRII.pdf}
}
Media sources and credits
Environment demonstrations and task schematics show the setting; the reported findings above come from the paper experiments.
SpringWorld. Project simulation and visualization; MuJoCo contributors. Two-body colors follow the paper renderer: object 1 is deep teal, object 2 is orange. All 201 archived states and their actions retain their original order, timing, geometry and camera; only the two display materials change. Playback follows simulation time, sampled every 0.05 s. Source.
PokeWorld. Project recordings, shown with the same orange object, gray pusher and light floor as the paper. All 24 stored states remain in source order, without interpolation or a new simulation. The viewing area is wider for the video so both complete trajectories stay visible. Action arrows show normalized inputs; the recorded disk geometry determines contact. This color visualization is separate from the model’s 64×64 observation renderer.
D-Clean. Project numerical trajectory and visualization. Preserves all 64 stored state and force samples. Black arrows show applied force; gray dashed arrows show −mγv. The final unavailable outgoing force remains unavailable rather than being replaced with zero. Playback is half physical speed.
Burgers. Burgers data: Zituo Chen · CC BY 4.0 · redrawn with RdBu_r. All 101 released numerical states retain their values and order. They span simulation time 0–5 s and are presented at 10 frames/s. Poster selects frame 50. Source trajectory SHA-256: 57f6aff9b412f3cd597f7c9d76ae300abc8a16a8607c6c85d07d9bc502a780dd. Source. License.
FitzHugh–Nagumo. Based on Zituo Chen’s DR2D generator · Python solver illustration. Existing Python reimplementation of the released equations, periodic stencil and RK4 integration, seed 42, k = 0.03, β = 0.20. Uses the upstream summer palette and fixed shared signed limits. Contains 101 states spanning 0–10 s at 10 frames/s; final-state poster. The separate software-record license is not inferred from the Burgers dataset license. Source. License.
Baxter tactile. Data: Youssef Amine, Christian Gianoglio and Maurizio Valle · CC BY 4.0. All 80 stored samples of the selected 16-channel grasp are retained, without smoothing or fabricated values. Playback reveals 10 samples per display second and holds the final state for one second; this is a display rate, not an acquisition-rate claim. Source. License.
RH20T. Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Chenxi Wang, Junbo Wang, Haoyi Zhu and Cewu Lu / RH20T. Demo: Hao-Shu Fang et al. / RH20T · CC BY-SA 4.0 · cropped and resized. Preserves all 260 consecutive frames and the original 30 frames/s presentation rate; no interpolation, speed changes or invented motion. Poster is source frame 225. Source original SHA-256: c0a02599ced9c907728d2d8dac8c7858fdee3f39835fbd74a4a137c63c29b8d1. Source. Reference. Reference. License.
Swimmer. Project simulation and visualization; MuJoCo contributors. The light floor and deep-teal body match the paper. The original three-link geometry, recorded trajectory and opposing joint torques are unchanged: rest (0–1 s), drive (1–4 s), reversal (4–7 s), then motors off (7–10 s). Every fourth stored state is shown at 25 frames/s, without interpolation. Grid, gray trail and torque labels are display overlays. Source.
Overcooked. Rendering adapted from JaxMARL contributors (Alexander Rutherford et al.). The archived 120-action sequence is replayed with the original environment and seed; all recorded positions, inventories, recipes and rewards match, and four original-renderer frames match the archived images pixel for pixel. Only display colors change: warm-white floor, pale counters, purple/gold partners and gray pots. All 121 frames retain their order and 120 ms display duration. The paper thumbnail uses the same rendered board. This is a UtilityGreedy controller demonstration, not a trained-policy result. Source. Reference. Reference.
Pendulum. CaDM: Kimin Lee, Younggyo Seo, Seunghyun Lee, Honglak Lee and Jinwoo Shin; Gym: OpenAI. Uses unchanged CaDM reset/step and normalization methods with a fixed input and seed 20260926. The original Gym geometry and colors were previously redrawn with Pillow; these are not native Gym pixels. Contains 161 real simulated states at 20 frames/s, with torque and response overlays. This curation only copies the pre-existing output. OpenAI Gym 0.16: MIT. CaDM: no top-level license identified in the existing source audit. Source. Reference. Reference.
CoPhy Collision. Original task schematic by the SPRII project, drawn from simple vector geometry. Arrows are explanatory motion cues. CoPhy benchmark.
CoPhy Balls. Original task schematic by the SPRII project, drawn from simple vector geometry. Arrows are explanatory motion cues. CoPhy benchmark.
CoPhy Blocktower. Original task schematic by the SPRII project, drawn from simple vector geometry. Arrows are explanatory motion cues. CoPhy benchmark.