Skip to content

perf(wan): batch-sharded cross-attn KV cache and batch-aware VAE mesh - #519

Open
Perseus14 wants to merge 1 commit into
mainfrom
perf/wan-batch-sharded-kv-cache-vae-mesh
Open

Perseus14 wants to merge 1 commit into
mainfrom
perf/wan-batch-sharded-kv-cache-vae-mesh

Conversation

@Perseus14

@Perseus14 Perseus14 commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Two memory fixes for Wan 2.2 T2V at per-device batch >= 1, from a B=1..16 batch-scaling sweep at 720p / 81 frames / 40 steps on v6e-8 and tpu7x-8.

1. Cross-attention KV cache is batch-sharded and scanned (transformer_wan.py)

compute_kv_cache ran an eager nnx.vmap over all 40 layers on replicated prompt embeds, so every device materialised the full CFG-doubled cache plus the FP32 RMSNorm temporaries for every layer at once (~6.8 GiB per video per device at 720p). It is now a single jitted executable that pins the batch axis to ('data', 'fsdp') on input and output and nnx.scans the layers one at a time. The cross-attention consumer already re-constrains K/V to (BATCH, CROSS_ATTN_HEAD, None, D_KV), so nothing is resharded at use time — each device simply holds its own B_local slice.

If the batch is not divisible by data * fsdp (e.g. data_parallelism > 1 with per_device_batch_size = 1 and CFG off — the same case the pipelines already tolerate for the CFG cache), the batch constraint is skipped and the cache stays replicated, as before. with_sharding_constraint does not raise on a non-divisible axis (JAX 0.11); skipping keeps that case explicit instead of depending on how the constraint treats it.

WanModel.compute_kv_cache is shared, so the i2v pipelines get the same cache layout. VACE has its own compute_kv_cache and is unchanged.

2. vae_spatial=-1 picks a batch-aware VAE mesh for T2V (wan_pipeline.py)

The VAE mesh is ('redundant', 'vae_spatial'); batch is sharded on redundant, and the longer spatial axis (height for portrait videos, width otherwise) / conv channels on vae_spatial. With the old auto value (vae_spatial = N) the batch axis is fully replicated, so at B = N every chip holds all N videos' activations and the causal conv stack pays ~10x (and OOMs on 32 GiB parts). Auto now resolves to (N, 1) — one video per device, no cross-chip collectives — when the global batch is a multiple of the device count, and to the unchanged (1, N) otherwise. Mixed meshes were measured and are not a win (at B=2 on v6e-8, (2, 4) needed a 21.2 GiB decode program vs 15.5 GiB for (1, 8)). An explicit vae_spatial still overrides.

This is on by default for T2V. The base configs ship vae_spatial: -1 and per_device_batch_size: 1.0, and the global batch is device count × per_device_batch_size, so any integer per-device batch selects (N, 1).

Scope: the batch-aware choice is gated by a new batch_parallel_vae argument on _create_common_components, enabled only by the T2V pipelines (wan_pipeline_2_1, wan_pipeline_2_2). The VAE constrains every activation to P('redundant', …, 'vae_spatial', …); on a (N, 1) mesh the batch axis is the only sharded one, so every tensor that goes through the VAE needs a batch divisible by N; _decode_latents_to_video raises ValueError otherwise, rather than silently decoding replicated on every chip. That holds for T2V (the only VAE call is the final full-batch decode) but not for VACE (reference images are encoded one at a time, [1, 1, H, W, C]) or the i2v / animate conditioning encodes. Those pipelines keep the previous (1, N) mesh, so nothing changes for them.

The mesh is picked from the config batch at load time, so calling a T2V pipeline directly with a batch that is not a multiple of the device count now raises on such a mesh (pass a full batch, or set vae_spatial explicitly). generate_wan.py always pads to the full batch.

Measurements (before rebasing onto current main; DP=2, CP=4, use_kv_cache=true unless noted)

Platform B Compile Denoise VAE decode (TPU) Peak HBM / device
tpu7x-8 2 main 229.9s 270.5s 2.2s 27.49 GiB
tpu7x-8 2 this PR 193.0s 269.8s 2.2s 16.65 GiB
tpu7x-8 8 main 265.8s 897.8s 41.6s 68.12 GiB
tpu7x-8 8 this PR 233.7s 893.4s 6.3s 24.32 GiB
v6e-8 (DP=1, FSDP=2, CP=4, vae_decode_chunk=1) 8 main — — — OOM in compute_kv_cache (6.25 GiB alloc, 4.99 GiB free)
v6e-8 (same) 8 this PR 192.2s 1302.6s 4.8s 17.41 GiB — completes

Denoise time is unchanged within noise; the wins are −64% peak HBM and 6.6x faster VAE decode at B=8 on tpu7x-8, and B=8 with the KV cache becoming possible at all on v6e-8.

Validation

New tests/wan/wan_vae_mesh_test.py (4 host devices on CPU):

  • _auto_vae_spatial table for B in {0,1,2,3,4,8,12,16} with the gate on and off.
  • A guard that only the T2V pipelines set batch_parallel_vae=True.
  • _batch_sharding / _shard_kv_cache_on_batch layout checks, including the non-divisible-batch fallback.
  • compute_kv_cache on a tiny scan_layers WanModel under the production logical axis rules, against the eager nnx.vmap it replaced: f32 and bf16, batch divisible and not divisible by data * fsdp. It also checks that the batch axis stays sharded and that the cached forward pass matches the uncached one.
  • _decode_latents_to_video rejecting a batch that does not divide the redundant axis.

Other notes:

  • generate_wan_smoke_test.py and wan_kv_cache_test.py called the T2V pipeline with a batch of 1 under the default per_device_batch_size; they now pass global_batch_size_to_train_on videos, as generate_wan.py does.
  • wan_kv_cache_test.py is TPU-only (attention=flash); not runnable on CPU before or after this change.
  • pyink clean. The branch is rebased onto current main, which fixed the two unrelated files the Linter workflow was failing on.

@Perseus14
Perseus14 requested a review from entrpn as a code owner October 25, 2026 11:22
@github-actions

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes memory usage and performance for the Wan model by replacing eager nnx.vmap with a jitted, scanned, and batch-sharded implementation (_scanned_compute_kv) for cross-attention KV cache computation. It also introduces dynamic vae_spatial mesh extent selection in the VAE pipeline to prevent OOMs and improve decoding speed. A critical issue was identified in the JIT compilation of _scanned_compute_kv, where blocks_graphdef must be marked as a static argument to avoid a JAX TypeError since it is not a valid JAX PyTree.

@Perseus14
Perseus14 force-pushed the perf/wan-batch-sharded-kv-cache-vae-mesh branch from 2a78845 to 182357d Compare October 25, 2026 20:23
@Perseus14 Perseus14 self-assigned this Oct 25, 2026
@Perseus14
Perseus14 force-pushed the perf/wan-batch-sharded-kv-cache-vae-mesh branch from 182357d to 98ea89f Compare October 25, 2026 23:18
@Perseus14
Perseus14 force-pushed the perf/wan-batch-sharded-kv-cache-vae-mesh branch from 98ea89f to 2b7e2bb Compare November 20, 2026 09:32
@Perseus14
Perseus14 requested a review from mbohlool November 20, 2026 11:44
@Perseus14
Perseus14 force-pushed the perf/wan-batch-sharded-kv-cache-vae-mesh branch from 6a91b43 to 4ac4bbf Compare November 20, 2026 12:29

@mbohlool mbohlool left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice memory win. Main ask: fail fast when the decode batch doesn't divide the batch-parallel VAE mesh.

@Perseus14
Perseus14 force-pushed the perf/wan-batch-sharded-kv-cache-vae-mesh branch 2 times, most recently from 06cd3dc to 947eb99 Compare November 20, 2026 22:00
mbohlool
mbohlool previously approved these changes Nov 21, 2026

@mbohlool mbohlool left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just one nit. LGTM otherwise.


def _decode_latents_to_video(self, latents: jax.Array, trace: Optional[dict] = None) -> np.ndarray:
"""Decodes latents to video frames and postprocesses."""
if self.vae_mesh is not None and hasattr(self.vae_mesh, "shape"):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: self.vae_mesh is always a Mesh (L520) and L1001 enters it right after, so the is not None and hasattr(self.vae_mesh, "shape") guard can go.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in fe4aa3d4 — dropped the guard; self.vae_mesh is always a Mesh there.

csgoogle
csgoogle previously approved these changes Nov 21, 2026
@Perseus14
Perseus14 dismissed stale reviews from csgoogle and mbohlool via 04f3eb7 November 21, 2026 19:16
@Perseus14
Perseus14 force-pushed the perf/wan-batch-sharded-kv-cache-vae-mesh branch 2 times, most recently from 04f3eb7 to fe4aa3d Compare November 21, 2026 19:21

@entrpn entrpn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice HBM and VAE decode speedup! Found two issues that need a quick fix before merging (plus one fail-fast suggestion):

  1. _scanned_compute_kv reads nn.logical_to_mesh_axes((BATCH,)) inside @jax.jit, where Flax's thread-local axis_rules context is not part of JAX's jit cache key.
  2. WanPipeline2_1 sets batch_parallel_vae=True, which also affects WanCheckpointer2_1 callers that run vae.encode (wan_txt2vid_data_preprocessing.py and data_processing_test.py).



@functools.partial(jax.jit, static_argnames=("num_layers", "mesh"))
def _scanned_compute_kv(blocks_graphdef, blocks_state, encoder_hidden_states, encoder_attention_mask, num_layers, mesh):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_scanned_compute_kv is decorated with @jax.jit, and _batch_sharding (L59) reads nn.logical_to_mesh_axes((BATCH,))[0] inside the traced function. Because nn_partitioning.axis_rules(...) is thread-local Python state and not part of JAX's jit cache key, calling compute_kv_cache once outside axis_rules (e.g., under bare with mesh:) or under different axis_rules caches an executable whose sharding is silently reused on subsequent calls with the same (blocks_graphdef, blocks_state, encoder_hidden_states, num_layers, mesh), leaving the KV cache replicated (P()) or sharded on the wrong mesh axes even when axis_rules is active.

Could we resolve batch_mesh_axes = nn.logical_to_mesh_axes((BATCH,))[0] (canonicalized to None or tuple[str, ...]) in compute_kv_cache before entering _scanned_compute_kv, and pass it via static_argnames=("num_layers", "mesh", "batch_mesh_axes")?

load_text_encoder=load_text_encoder,
load_scheduler=load_scheduler,
# T2V only runs the VAE on the final full-batch decode, so the
# batch-aware (N, 1) mesh is safe here; see _auto_vae_spatial.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WanCheckpointer2_1 instantiates WanPipeline2_1, which is also used for vae.encode in src/maxdiffusion/data_preprocessing/wan_txt2vid_data_preprocessing.py:147 and src/maxdiffusion/tests/data_processing_test.py:74 under default base_wan_14b.yml (vae_spatial: -1, per_device_batch_size: 1.0 -> global_batch_size_to_train_on == N -> vae_mesh = (N, 1)):

  • wan_txt2vid_data_preprocessing.py:118-127 encodes chunks of batch_size = 10 (plus any trailing remainder), which raises XLA IndivisibleError inside WanCausalConv3d when redundant is 4 or 8.
  • data_processing_test.py:74-89 encodes a single video (batch = 1), which degrades P('redundant', ...) to P() and runs the full 81-frame video encode replicated on every chip instead of spatially sharded on (1, N).

Those encode/checkpoint paths should keep a full-spatial (1, N) VAE mesh (and AutoencoderKLWan.encode / vae_encode_pass may also want the same redundant divisibility guard as _decode_latents_to_video so batch=1 on a (N, 1) mesh fails fast rather than encoding replicated).

redundant = self.vae_mesh.shape.get("redundant", 1)
if redundant > 1 and latents.shape[0] % redundant != 0:
raise ValueError(
f"VAE decode batch size ({latents.shape[0]}) must be divisible by the 'redundant' mesh axis "

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional (fail-fast UX): Since base_wan_*.yml defaults to per_device_batch_size: 1.0 and vae_spatial: -1 (redundant == total_devices), a direct Python call like pipeline(prompt="...") (batch_size = 1, output_type="np") will currently run all 40-50 denoising steps before hitting this ValueError at decode time. Checking redundant divisibility up front in check_inputs / _prepare_model_inputs when output_type != "latent" (in addition to keeping the guard here) would fail fast before denoising.

Two memory fixes for Wan 2.2 T2V at per-device batch >= 1, motivated by a
batch-scaling sweep (B=1..16) at 720p/81f/40 steps on v6e-8 and tpu7x-8.

1. Cross-attention KV cache (transformer_wan.py)

   compute_kv_cache ran an eager nnx.vmap over all 40 layers on replicated
   prompt embeds, so every device materialised the full CFG-doubled cache
   plus the FP32 RMSNorm temporaries for every layer at once. It is now one
   jitted executable that pins the batch axis to ('data', 'fsdp') on input
   and output and scans the layers one at a time. The cross-attn consumer
   already re-constrains K/V to that layout, so nothing is resharded at use
   time; each device just holds its own B_local slice.

   If the batch is not divisible by data * fsdp (e.g. data_parallelism > 1
   with per_device_batch_size = 1 and CFG off, the case the pipelines
   already tolerate for the CFG cache) the batch constraint is skipped and
   the cache stays replicated as before. with_sharding_constraint does not
   raise on a non-divisible axis (JAX 0.11); skipping keeps that case
   explicit instead of depending on how the constraint treats it.

   WanModel.compute_kv_cache is shared, so the i2v pipelines get the same
   cache layout. VACE has its own compute_kv_cache and is unchanged.

   Sharding the 512-token text axis on 'context' as well was tried and
   rejected: XLA hoists the per-layer all-gather out of the layer scan, so
   the forward program's reservation grew by 5.8 GiB (net loss on v6e-8).

2. VAE mesh (wan_pipeline.py), T2V pipelines only

   With vae_spatial=-1 (auto) the VAE mesh is now ('redundant', 'vae_spatial')
   = (N, 1) when the global batch is a multiple of the device count N, i.e.
   one video per device with no cross-chip collectives, and (1, N) otherwise
   (unchanged). Mixed meshes were measured and are not a win: at B=2 on
   v6e-8 a (2, 4) mesh needed a 21.2 GiB decode program vs 15.5 GiB for
   (1, 8). An explicit vae_spatial still overrides.

   This is on by default for T2V: the base configs ship vae_spatial: -1 and
   per_device_batch_size: 1.0, and the global batch is device count *
   per_device_batch_size, so any integer per-device batch selects (N, 1).

   The batch-aware choice is gated by a new batch_parallel_vae argument on
   _create_common_components, enabled only by wan_pipeline_2_1 / 2_2. The
   VAE constrains activations to P('redundant', ..., 'vae_spatial', ...); on
   a (N, 1) mesh the batch axis is the only sharded one, so every tensor
   through the VAE needs a batch divisible by N (_decode_latents_to_video
   raises ValueError otherwise, rather than silently decoding replicated on
   every chip). That holds for T2V (only the final full-batch decode)
   but not for VACE (reference images encoded one at a time,
   [1, 1, H, W, C]) or the i2v / animate conditioning encodes; those keep
   the (1, N) mesh.

Tests: new tests/wan/wan_vae_mesh_test.py (4 host devices on CPU):
- the _auto_vae_spatial table for B in {0..16}, gated and ungated;
- a guard that only the T2V pipelines set the gate;
- _batch_sharding / _shard_kv_cache_on_batch layouts including the
  non-divisible fallback;
- compute_kv_cache on a tiny scan_layers WanModel under the production
  logical axis rules against the eager nnx.vmap it replaced (f32 and bf16,
  batch divisible and not divisible by data * fsdp), including that the
  batch axis stays sharded and that the cached forward matches the
  uncached one.
- _decode_latents_to_video rejecting a batch that does not divide the
  'redundant' axis.

generate_wan_smoke_test.py and wan_kv_cache_test.py called the T2V pipeline
with a batch of 1 under the default per_device_batch_size; they now pass
global_batch_size_to_train_on videos, as generate_wan.py does.

Measured before rebasing onto current main (DP=2, CP=4 unless noted,
use_kv_cache=true):

  tpu7x-8  B=2  peak HBM 27.49 -> 16.65 GiB, denoise 270.5 -> 269.8 s
  tpu7x-8  B=8  peak HBM 68.12 -> 24.32 GiB, VAE decode 41.6 -> 6.3 s,
                denoise 897.8 -> 893.4 s
  v6e-8    B=8  (DP=1, FSDP=2, CP=4, vae_decode_chunk=1) main OOMs in
                compute_kv_cache (6.25 GiB alloc, 4.99 GiB free); with this
                change it completes: peak 17.41 GiB, denoise 1302.6 s,
                VAE decode 4.8 s.
@Perseus14
Perseus14 force-pushed the perf/wan-batch-sharded-kv-cache-vae-mesh branch from fe4aa3d to 6347d1a Compare November 21, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants