Latent Interactive World Models

ω-EVA

Envision, Verify, and Act with Latent Interactive World Models

Zhenguo Sun^*, Yu Sun^*†, Hande Huang, and Alois Knoll

^*Equal contribution. ^†Project lead.

Paper Code

Latent Consequence

01Propose

02Envision

03Verify

04Act

Reason about what an action will cause before committing it to the robot.

The Core Idea

A policy should inspect its imagined consequence before it acts.

Embodied policies usually map the current observation directly to an action, leaving candidate-action consequences implicit. ω-EVA closes this gap with an Envision–Verify–Act loop: a policy proposes an action chunk, an action-conditioned latent world model predicts the future induced by that proposal, and a tri-branch refiner jointly reasons over the current state, imagined future, and proposed action. Consequence reasoning remains in feature space, so no future video is generated during inference.

Envision · Verify · Act

The world model becomes an action-feedback module.

Rather than serving only as a training objective or passive predictor, the world model interacts with the policy's own proposal inside a single control decision.

Propose

A language-conditioned flow policy generates an initial action chunk from dynamics-aware visual features.

Envision

The proposal is fed back into the latent world model to predict its action-conditioned future.

Verify

A tri-branch refiner compares the present, predicted consequence, and proposal to rewrite the action.

Act

The refined action is executed, then the loop restarts from the next observation.

Three-Stage Framework

Learn dynamics, generate a proposal, then close the loop.

Overview of the omega-EVA three-stage architecture and Envision Verify Act loop — Stage 1 learns action-conditioned latent dynamics. Stage 2 trains a world-aware flow policy. Stage 3 feeds the policy's proposal back through the frozen world model and refines it from the imagined consequence.

Interactive Consequence Reasoning

Refinement changes both the action and the future it induces.

The same scene is shown for the Stage 2 proposal and the Stage 3 refined action. The imagined futures visualize latent predictions through a diagnostic decoder.

Stage 2 Proposal policy

Stage 2 imagined future — Imagined future

Stage 3 Envision–Verify–Act

Stage 3 refined action rollout — Refined rollout

Stage 3 imagined future — Refined future

Diagnostic visualizations only. ω-EVA performs consequence reasoning in latent feature space and does not decode videos during policy inference.

Experimental Results

Compact, competitive, and consistently improved by Stage 3.

Evaluated across single-arm, bimanual, long-horizon, and perturbed simulation settings, with no additional robot-data pretraining.

LIBERO

98.6%

97.9+0.7 Stage 3

LIBERO-PLUS

83.4%

81.2+2.2 benchmark training

RoboTwin 2.0

90.3%

88.9+1.4 Stage 3

Core model scale

≈1.2B

0 robot-data pretraining

Stage 2 → Stage 3 across benchmarks

Task success rate (%). Higher is better.

Benchmark	Training setting	Stage 2	Stage 3	Gain
LIBERO	Benchmark data	97.9	98.6	+0.7
LIBERO-PLUS	Zero-shot from LIBERO	71.3	72.2	+0.9
LIBERO-PLUS	Benchmark data	81.2	83.4	+2.2
RoboTwin 2.0	Benchmark data	88.9	90.3	+1.4

Representation Diagnostics

Future prediction focuses visual features on interaction.

Dynamics-aware features shift activation toward the end effector, manipulated object, and nearby regions whose relationships can change under action.

Current robot observation — 01Current observation

DINOv3 activation heatmap — 02Frozen DINOv3

Stage 1 activation heatmap — 03Stage 1 c_t

Stage 2 activation heatmap — 04Co-trained c_t

92.1 → 97.9 Average LIBERO success from raw DINOv3 features to co-trained c_t.

Δ = 0 Current representation remains identical under expert, dummy, random, and shuffled actions.

Action-conditioned Only the future branch observes the candidate action by design.

Latent Future Fidelity

The refined action imagines a future closer to the target.

A single diagnostic decoder makes predicted DINOv3 futures observable. It is used only for analysis, never for policy training or inference.

Decoded ground truth future — Decoded GT

Ground truth action imagination — GT-action imagination

Stage 2 proposal imagined future — Stage 2 proposal

Stage 3 refined action imagined future — Stage 3 refined

Paired structural fidelity

SSIM 0.9520 → 0.9562 Refined-action futures align more closely with the paired target.

Set-level distribution

FID 7.41 → 7.40 Distributional fidelity is preserved with a slight improvement.

Mechanism Ablation

Future feedback and proposal anchoring are both necessary.

The imagined consequence supplies feedback. The proposal identifies the action that must be corrected. Removing either branch falls below the complete Stage 3 model.

Current Future Proposal

98.6

Full Stage 3

97.2

w/o imagined future

96.0

w/o action proposal

Citation

Build on omega-EVA.

The code is now available on GitHub. Dataset releases are being prepared, and the paper is publicly available on arXiv.

Read the paper

@article{sun2026omegaeva,
  title   = {$\omega$-EVA: Envision, Verify, and Act
             with Latent Interactive World Models},
  author  = {Sun, Zhenguo and Sun, Yu and
             Huang, Hande and Knoll, Alois},
  journal = {arXiv preprint arXiv:2606.09457},
  year    = {2026}
}