Align and expand official LTX-2 / LTX-2.3 pipeline support

Motivation

LTX-2 and LTX-2.3 share most model components, but their official inference recipes differ in checkpoint composition, denoising schedule, guidance, output-resolution semantics, adapter state, and audio ownership. This issue tracks those recipes as explicit, independently testable contracts built on one shared runtime.

The shared runtime landed in #5147, followed by official one-stage alignment in #5148. Two-stage and HQ recipes are active WIP in #5500; specialized pipelines are tracked separately below.

Core recipe status

Status: ✅ supported in main; 🚧 active WIP with an owner and PR; ❌ not supported.

Official recipe	Essential execution and checkpoint contract	Status / PR	Remaining work
Dev one-stage	Run the version-matched dev checkpoint at the requested output resolution with official multimodal guidance and schedule	✅ LTX-2/LTX-2.3 — #5148	Extend accuracy and runtime-feature coverage as new backends are enabled
Standard two-stage	Run dev at half resolution; spatially upsample; refine video for three positive-only steps with the same dev model plus distilled LoRA; keep Stage 1 audio	🚧 LTX-2/LTX-2.3 WIP — #5500	Merge the recipe and its accuracy/runtime-feature coverage
HQ two-stage	LTX-2.3; apply the distilled LoRA in both stages with recipe-specific strengths; use the second-order sampler and HQ defaults; keep Stage 1 audio	🚧 LTX-2.3 WIP — #5500	Merge the sampler, per-stage LoRA strengths, T2V/I2V, and accuracy coverage
Full-distilled two-stage	Use the full-distilled model in both stages with fixed 8-step and 3-step positive-only schedules around spatial upsampling; return Stage 2 video and audio	✅ LTX-2 — #5148🚧 LTX-2.3 WIP — #5500	Merge LTX-2.3 support and broaden accuracy coverage

Checkpoint and distillation notes

• Dev one-stage requires the version-matched dev components and text encoder.
• Standard and HQ use a dev base, distilled LoRA, and spatial latent upsampler; these roles must be loadable from separate sources.
• Full-distilled uses a full-distilled model and spatial latent upsampler. It is a different weight contract from dev plus distilled LoRA.
• Distillation does not reduce Transformer parameter count. It reduces inference compute through shorter schedules and positive-only prediction. Quantization is a separate optimization for weight size and memory.
• Two-stage request dimensions always describe the final output resolution.

Implementation constraints

• All recipes share the common runtime but keep independently testable schedule, guidance, adapter, checkpoint, and output contracts.
• Every phase explicitly establishes its complete guidance and adapter state; state from a previous phase or request must not leak.
• T2V and I2V share stage orchestration; conditioning changes request preparation rather than checkpoint selection.
• Two-stage width and height consistently mean final output dimensions.
• Existing distributed execution, offload, compilation, caching, batching, and serving behavior must either be preserved or rejected through explicit, tested capability checks.

LTX pipelines × diffusion features

Status: ✅ fully supported in main; 🚧 implemented in an active PR; ❓ not verified yet; ❌ not supported. Each cell covers both T2V and I2V where available. Standard and HQ cover both dynamic and resident LoRA modes.

Pipeline	⚡Cache-DiT	🔀SP (Ulysses & Ring)	🔀Tensor-Parallel	🔀HSDP	💾CPU Offload (Module/Layerwise)	💾VAE-Patch-Parallel	🔄Request Batching
Dev (2/2.3)	✅	✅	✅	✅	✅	✅	✅
Full-distilled (2/2.3)	❌	🚧	🚧	🚧	🚧	🚧	❌
Standard (2/2.3)	❌	🚧	🚧	🚧	🚧	🚧	❌
HQ (2.3)	❌	🚧	🚧	🚧	🚧	🚧	❌

Other specialized official pipelines

These pipelines reuse the LTX component graph and should extend the shared runtime and recipe abstractions rather than introduce independent model implementations. A WIP row must identify its owner and active PR; after merge, retain the PR for provenance and change the status to supported.

Official specialized pipeline	Purpose	Status / owner / PR	Implemented scope
Text-to-audio one-stage	Generate audio without video	❌ Not supported	—
Keyframe interpolation	Generate between supplied keyframes	🚧 One-stage WIP — @lucasbarrez, #5457❌ Two-stage	LTX-2/LTX-2.3 one-stage FLF2V/FMLF pipeline, multi-anchor Video API, CPU tests, and manual H100 E2E validation
Audio-to-video two-stage	Generate video while preserving input audio	❌ Not supported	—
IC-LoRA pipeline	Image/video-controlled generation	❌ Not supported	—
Retake	Regenerate a selected temporal region	❌ Not supported	—
HDR IC-LoRA	Produce HDR video from a reference	❌ Not supported	—
LipDub	Re-voice/lip-sync using video and audio references	❌ Not supported	—