- Quantization reduces model weight precision to lower VRAM requirements for local inference. LTX-2.5 supports FP8 natively, cutting VRAM by around 40% with minimal quality loss, via either fp8-cast (any compatible GPU) or fp8-scaled-mm (Hopper GPUs with TensorRT-LLM).
- MXFP8 improves on standard FP8 by applying block-level scaling to preserve more dynamic range, while NVFP4 halves memory again to roughly 11GB for a 22B model. Both are tied to NVIDIA Blackwell architecture.
- For most local deployments today, start with fp8-cast combined with the distilled checkpoint to run LTX-2.5 on a 32GB GPU at production quality.
Running a 22-billion-parameter video model on a local GPU means every byte of memory counts. Full-precision weights demand more VRAM than most consumer and even many professional cards can offer, and that gap between model size and available memory is where quantization becomes essential. Choosing the right numeric format for inference determines whether you can run locally at all, how fast each generation completes, and how much visual quality you retain.
This guide explains three quantization formats relevant to AI video inference: FP8, MXFP8, and NVFP4. It covers what each format is, how they compare in precision and memory savings, and how to configure FP8 quantization in LTX-2.5 today.
What Is Quantization in AI Video Generation?
Quantization reduces the numeric precision of model weights from higher-bit formats (FP32 or FP16) to lower-bit representations. For video generation models this is more consequential than for image or text models, because video adds a temporal dimension. A single generation pass processes hundreds of spatial-temporal tokens, and every weight stored at full precision multiplies VRAM consumption by frame count and resolution.
How Numeric Precision Affects VRAM and Speed
A model weight stored in FP32 occupies 4 bytes. FP16 cuts that to 2 bytes. FP8 halves it again to 1 byte, and FP4 formats push down to 0.5 bytes per weight. For a model with billions of parameters, these reductions translate directly to gigabytes of freed VRAM. The trade-off is always precision: fewer bits mean a narrower range of representable values, which can introduce rounding errors that affect generation quality.
In practice, the quality impact depends on which layers are quantized and how. Quantizing transformer linear layers (where most parameters live) while keeping critical components like the VAE and text encoder at full precision is the standard approach for preserving output quality while reducing the memory footprint substantially.
Why Video Models Are Different from LLMs
Large language models process 1D token sequences. Video diffusion models process 3D spatial-temporal latents, and in LTX-2.5's case generate audio alongside the picture, with the two streams decoded through separate VAEs. Memory pressure is higher because the denoising loop runs multiple forward passes per generation step, and two-stage pipelines add more by generating at low resolution first and then upsampling. Quantization is not optional for local deployment. It is a prerequisite.
FP8 Quantization Explained
FP8 (8-bit floating point) is the most mature and widely supported reduced-precision format for AI inference. It stores each weight in a single byte using two standard variants: E4M3 (4 exponent bits, 3 mantissa bits) optimized for weights and activations, and E5M2 (5 exponent bits, 2 mantissa bits) optimized for gradients. For inference, E4M3 is the relevant variant because it preserves more precision in the value range that matters for model weights. In LTX-2.5, enabling FP8 quantization reduces VRAM usage by approximately 40% with minimal quality loss.
FP8 Cast vs FP8 Scaled Matrix Multiplication
LTX-2.5 supports two FP8 quantization policies, each suited to different hardware and use cases.
FP8 Cast downcasts transformer linear weights to FP8 during model loading and upcasts them back to higher precision on the fly during each inference step. This approach requires no extra dependencies and works on any GPU with FP8 support. It reduces the memory footprint for weight storage while keeping computation in higher precision.
FP8 Scaled Matrix Multiplication uses NVIDIA TensorRT-LLM's scaled matrix multiplication for efficient FP8 math. Weights are stored in FP8 with per-tensor scaling, and inputs are quantized dynamically, or statically with a calibration file. This backend is optimized for Hopper-architecture GPUs and delivers both memory savings and faster computation by running the actual math in FP8.
How to Enable FP8 in LTX-2.5
Both policies require setting the PYTORCH_CUDA_ALLOC_CONF environment variable to enable expandable memory segments. Here is the CLI configuration for each:
FP8 Cast (works on any GPU with FP8 support):
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True python -m ltx_pipelines.ti2vid_two_stages --quantization fp8-cast --checkpoint-path /path/to/checkpoint.safetensors
FP8 Scaled MM (requires TensorRT-LLM, best on Hopper GPUs):
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True python -m ltx_pipelines.ti2vid_two_stages --quantization fp8-scaled-mm --checkpoint-path /path/to/checkpoint.safetensors
For programmatic usage, pass a QuantizationPolicy when building the pipeline from its config:
python
from ltx_core.quantization.policy import QuantizationPolicy
pipeline = DistilledPipeline.from_config(
"path/to/config.yaml",
quantization=QuantizationPolicy.fp8_cast(),
)
Swap in QuantizationPolicy.fp8_scaled_mm() for the Hopper-optimized backend.
MXFP8: Microscaling for Better Accuracy
MXFP8 (Microscaling FP8) is an industry-standard format defined by the Open Compute Project's Microscaling Formats Specification. While standard FP8 applies a single scale factor per tensor or per channel, MXFP8 applies scale factors at the block level, typically to groups of 32 elements. This finer-grained scaling preserves more dynamic range within each block, reducing the quantization error that accumulates in layers with highly variable weight distributions.
MXFP8 vs FP8: When Microscaling Matters
The practical difference shows up in layers where weight magnitudes vary significantly within a single tensor. Standard per-tensor scaling forces all values to share one scale factor, which can clip outliers or waste precision on small values. Block-level microscaling lets each group of 32 elements use its own scale, preserving both outliers and fine detail. For a model that carries picture and audio through the same generation, that granularity matters most in the cross-attention layers, where weight distributions differ between the two modalities.
MXFP8 hardware support is available on NVIDIA Blackwell-architecture GPUs (B100, B200, GB200). As MXFP8 support expands in the PyTorch ecosystem, it represents a path toward better quality at the same 8-bit memory footprint.
NVFP4: NVIDIA's 4-Bit Floating Point Format
NVFP4 is NVIDIA's 4-bit floating-point format, introduced with the Blackwell GPU architecture. At half a byte per weight, NVFP4 roughly halves the memory footprint compared to FP8, letting models that were previously GPU-memory-bound fit on smaller hardware or run larger batch sizes.
NVFP4 Inference Speed and VRAM Reduction
The memory savings are significant. A 22B-parameter model stored in FP8 requires roughly 22GB for weights alone. NVFP4 brings that to around 11GB. Combined with 4-bit Tensor Core operations on Blackwell GPUs, both memory bandwidth and compute throughput improve. For local inference where VRAM is the primary constraint, this opens the possibility of running large video models on hardware that previously required FP8 or FP16 to function.
Trade-offs: When 4-Bit Is and Isn't Enough
Four bits provide only 16 discrete values per weight. Even with block-level scaling, the reduced precision introduces more quantization noise than 8-bit formats. For video generation, where temporal consistency depends on precise weight values across frames, 4-bit quantization may produce visible artifacts in smooth motion sequences or fine texture detail. Suitability depends on the specific pipeline, resolution, and quality requirements.
An official Lightricks NVFP4 checkpoint is published on HuggingFace for LTX-2.3, built using quantization-aware distillation, which gives a useful indication of how the format behaves on this architecture.
Comparing FP8, MXFP8, and NVFP4 for AI Video
Which Format Should You Use?
If you are running LTX-2.5 locally today, FP8 is the practical choice and the only quantization format supported natively in the codebase. The fp8-cast policy works on any GPU with FP8 support and requires no additional dependencies. If you have access to a Hopper GPU and need maximum throughput, fp8-scaled-mm with TensorRT-LLM delivers native FP8 computation. MXFP8 and NVFP4 are forward-looking formats tied to Blackwell hardware, with MXFP8 support continuing to expand across the PyTorch ecosystem.
Combining Quantization with Other Optimizations
FP8 quantization is most effective alongside the other levers LTX-2.5 exposes for constrained hardware.
Use the distilled checkpoint. It runs 8 steps where the full model takes 20 to 50, which cuts both time and peak memory. On a 32GB card this is the difference between working and not.
Enable tiled VAE decode. Decoding splits into tiles to reduce peak memory at the cost of slightly slower decoding. This is the fix when out-of-memory errors happen at the decode stage rather than during sampling, and quantization alone does not address it.
Offload text encoding to the API. The Gemma 4 text encoder and prompt enhancer both occupy VRAM. Running them through the LTX API instead of locally frees meaningful headroom for generation.
Combining FP8 quantization with the distilled checkpoint gives a pipeline that runs on a 32GB GPU at production quality, which is the documented minimum configuration.
Getting Started
If you are setting up LTX-2.5 for local inference, start with FP8 Cast quantization. It requires no additional dependencies, works across GPU generations with FP8 support, and provides meaningful VRAM reduction without quality compromise.
Before starting, ensure your system has CUDA 13.2 or newer installed. This is a hard prerequisite, alongside Python 3.10+ and PyTorch 2.7.
- Clone the repository and install dependencies:
git clone https://github.com/Lightricks/LTX-2.git && cd LTX-2 && uv sync --frozen - Download the distilled checkpoint, the Gemma 4 text encoder, the video and audio VAEs, and the spatial upscaler from the LTX-2.5 HuggingFace repository
- Run your first quantized generation with the
--quantization fp8-castflag - Monitor VRAM usage and adjust resolution and frame count to fit your hardware. Frame counts must follow the
(F-1) % 8 == 0constraint, so valid values include 57, 65, 97, and 121. Dimensions must be divisible by 64 for two-stage pipelines and 32 for one-stage
The quantization format you choose is one piece of a larger optimization stack. Combine it with the right pipeline selection, sampling configuration, and hardware-aware settings to get the most from local AI video generation.
