Abstract
We present RynnWorld-4D, a novel 4D embodied world model that shifts the paradigm from 2D pixel prediction to physically grounded 4D scene evolution. By co-generating synchronized RGB, depth, and optical flow (RGB-DF), RynnWorld-4D captures the underlying 3D geometry and temporal motion trajectories, creating a representation space that effectively bridges the gap between generative world modeling and low-level robotic control. We release RynnWorld-4D as a unified diffusion framework featuring a specialized tri-branch architecture, alongside RynnWorld-4D‑Policy (an inverse dynamics head for high-frequency, closed-loop bimanual manipulation).
🌟 Key Highlights
-
Projective 4D Representation:
Integrates RGB, Depth, and Optical Flow into a unified RGB-DF format, allowing pixels to be unprojected into metric 3D scene flow for precise geometric and kinetic grounding. -
Tri-branch Diffusion Architecture:
Employs three dedicated transformer branches with mutual cross-modal attention, ensuring that appearance, geometry, and motion evolve with high spatio-temporal consistency. -
Action-from-Latent Policy:
RynnWorld-4D-Policy bypasses expensive multi-step denoising by directly consuming internal 4D latents, enabling high-frequency (9Hz+), closed-loop dexterous control.
Overview
RynnWorld-4D
Real-world experiments
Click on the video to view a larger version.
Dual Picking
Dual Picking
Block Pushing
Hand-over
Bimanual Lifting
Bimanual Lifting
Lid Placement
Lid Placement
Bowl Stacking
Bowl Stacking
BibTeX
@article{RynnWorld-4D2026,
title = {RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation},
author = {Haoyu Zhao and Xingyue Zhao and Siteng Huang and Xin Li and Deli Zhao and Zhongyu Li},
year = {2026}
}