WorldAttention

An Efficient Attention Architecture
for Interactive Video World Models

Paper Code

scroll down to explore

WorldAttention: An Efficient Attention Architecture for Interactive Video World Models

NeurIPS 2026 Reviewed (5/5/5 - Accept/Accept/Accept)

Zeyu Zhang^1* Jinyuan Mao^2* Dakai An^3* Wangbo Zhao³ Hanfeng Lu³ Jiasheng Tang^1,4† Yinghao Yu⁵ Wei Wang³ Bohan Zhuang^1,2†

¹DAMO Academy, Alibaba Group ²Zhejiang University ³HKUST ⁴Hupan Lab ⁵TRE, Alibaba Group

^*Equal contribution. ^†Corresponding authors.

Paper Code

TL;DR: WorldAttention enables efficient interactive video world models through hybrid sparse attention, hierarchical KV caching, and specialized kernels, achieving 22 FPS.

Ironman

WorldAttention

0:00 / 0:00

DeepForcing

0:00 / 0:00

RollingForcing

0:00 / 0:00

SkyReels-V2

0:00 / 0:00

Infinity-RoPE

0:00 / 0:00

LongLive

0:00 / 0:00

Cyberpunk Workshop

WorldAttention

0:00 / 0:00

DeepForcing

0:00 / 0:00

RollingForcing

0:00 / 0:00

SkyReels-V2

0:00 / 0:00

Infinity-RoPE

0:00 / 0:00

LongLive

0:00 / 0:00

Poker

WorldAttention

0:00 / 0:00

DeepForcing

0:00 / 0:00

RollingForcing

0:00 / 0:00

SkyReels-V2

0:00 / 0:00

Infinity-RoPE

0:00 / 0:00

LongLive

0:00 / 0:00

Christmas Street

WorldAttention

0:00 / 0:00

DeepForcing

0:00 / 0:00

RollingForcing

0:00 / 0:00

SkyReels-V2

0:00 / 0:00

Infinity-RoPE

0:00 / 0:00

LongLive

0:00 / 0:00

Stage Dance

WorldAttention

0:00 / 0:00

DeepForcing

0:00 / 0:00

RollingForcing

0:00 / 0:00

SkyReels-V2

0:00 / 0:00

Infinity-RoPE

0:00 / 0:00

LongLive

0:00 / 0:00

Workbench Demo

WorldAttention

0:00 / 0:00

DeepForcing

0:00 / 0:00

RollingForcing

0:00 / 0:00

SkyReels-V2

0:00 / 0:00

Infinity-RoPE

0:00 / 0:00

LongLive

0:00 / 0:00

Method

WorldAttention overall architecture combining Hierarchical KV Cache retrieval and Hybrid Sparse Attention. — Overall architecture of WorldAttention. We propose a system-oriented co-design using Hierarchical KV Cache (HKV) for coarse-to-fine memory retrieval, and Hybrid Sparse Attention (HSA) for efficient dual-branch attention computation.

WorldAttention three-tier hierarchical KV cache with prompt-level and page-level retrieval. — Overview of the WorldAttention architecture featuring a three-tier hierarchical KV cache and a two-stage (prompt-level and page-level) retrieval mechanism for efficient long-video generation.