- Match the GPU to the workload, not the other way around: preview iteration, production rendering, LoRA training, and full fine-tuning each have a different VRAM floor — buying "the biggest card" often means paying for headroom you don't use for your actual job.
- Concrete tiers: 16GB covers preview/iteration, 24–32GB (RTX 4090/5090) is the sweet spot for solo production work, 48GB unlocks practical LoRA fine-tuning, and 80GB+ (A100/H100) is where full model training and audio-video LoRA at full sequence length become tractable.
The right GPU for AI video is the one that fits the job you actually do — not the biggest card you can buy. A card that generates a decent 5-second preview in 90 seconds is wasted if you spend all day fine-tuning a LoRA that keeps hitting OOM.
Match the hardware tier to the workload, and everything downstream — iteration speed, cost per generation, batch throughput — gets easier.
This guide maps four common AI video workloads to concrete GPU picks, grounded in what LTX-2.3 actually requires at each tier, with FP8 quantization and gradient estimation factored in.
Why video hardware isn't image hardware
Video models hold more state in VRAM than image models because temporal information must survive every denoising step. LTX-2 is an asymmetric dual-stream diffusion transformer with a 14B-parameter video stream and a 5B-parameter audio stream sharing 48 transformer blocks.
That means two latent tensors, two RoPE encodings, and cross-modal attention between them — all live in GPU memory during inference. A card that runs Stable Diffusion at 1024×1024 without breaking a sweat can still OOM on a 121-frame LTX-2.3 clip.
What actually consumes VRAM
VRAM usage in LTX-2 scales with four inputs. Resolution: 720p to 1080p is roughly a 2.25× memory jump, 1080p to 4K is roughly 4×. Frame count: the Video VAE compresses on a fixed rule — frame count must satisfy (F-1) % 8 == 0, so valid counts at 25fps are 9, 17, 25, 33, 41, 49, 57, 65, 73, 81, 89, 97, 105, 113, 121.
Each jump roughly doubles memory. Active control mechanisms: IC-LoRAs, keyframe conditioning, and audio guidance all add memory. Pipeline stages: two-stage pipelines hold multiple latent representations simultaneously.
What FP8 changes
FP8 quantization is the single largest lever between "won't run" and "runs comfortably." LTX-2 ships with two policies: fp8-cast downcasts transformer linear weights to FP8 during loading and upcasts on the fly during inference (works on any GPU with FP8 support), and fp8-scaled-mm uses TensorRT-LLM's cublas_scaled_mm for FP8 matrix multiplication (best on Hopper GPUs).
Either policy is combined with PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to reduce fragmentation. In practice, FP8 pulls a comfortable-VRAM tier down by roughly one card generation.
Gradient estimation is the other major lever, this time on the compute side rather than memory: it reduces inference steps from 40 to 20–30 while maintaining quality, so it compounds well with FP8 rather than competing with it for the same headroom.
The four AI video workloads that determine your GPU
Before naming cards, name jobs. Every practical GPU decision in AI video generation falls into one of four buckets — and the answer changes for each.
Workload 1: Preview and prompt iteration
Preview iteration is short clips at draft resolution, run dozens of times a day while a prompt or reference image gets refined. The pipeline choice is DistilledPipeline — 8 predefined sigmas in stage 1, 4 in stage 2, no guidance, fastest inference of any LTX-2 pipeline. Or TI2VidOneStagePipeline at 512×768 for quick prototyping.
Workload 2: Production-quality single-shot generation
Production generation is a completed shot at full resolution, ready to hand off. The pipeline is TI2VidTwoStagesPipeline — stage 1 generates at low resolution with multimodal guidance, stage 2 upsamples 2× with a distilled LoRA. It's the recommended pipeline for production quality.
Or TI2VidTwoStagesHQPipeline with the res_2s second-order sampler for fewer steps at equivalent quality.
Workload 3: LoRA fine-tuning on a single card
LoRA fine-tuning trains lightweight adapter layers to teach a style, camera behavior, or motion pattern. LTX-2's trainer supports Standard LoRA on a single GPU, plus IC-LoRA (in-context) training for paired reference-target sequences that unlocks video-to-video capabilities like deblurring, colorization, and pose control.
A low-VRAM training variant with INT8 quantization is documented for 32GB GPUs.
Workload 4: Full training and multi-GPU workflows
Full model fine-tuning gives maximum flexibility on multi-GPU setups but demands substantial compute — the kind of workload where enterprise cards earn their price.
This tier also handles audio-video LoRA (joint conditioning while preserving synchronization) and the specialized IC-LoRA training patterns that need larger batches.
GPU recommendations by use case
Here's the mapping from workload to hardware, calibrated to LTX-2.3's actual requirements.
16 GB tier — RTX 5080, RTX 4080 SUPER
Sixteen-gigabyte cards are the practical floor for LTX-2 preview iteration with FP8 quantization enabled. Expect draft-resolution single-shots, short clips (33–49 frames), DistilledPipeline or single-stage with quantization active.
IC-LoRA workflows are possible but require running only one control group at a time. Not enough headroom for two-stage production at 1080p+ without aggressive optimization.
24–32 GB tier — RTX 4090, RTX 5090
The RTX 4090 (24 GB) and RTX 5090 (32 GB) are the sweet spot for solo production work. Both handle TI2VidTwoStagesPipeline at target resolution with FP8 quantization, run ICLoraPipeline for video-to-video with reference conditioning, and generate LTX-2.3 content up to 97-frame clips (4-second content at 25fps) without heavy juggling.
The 5090's extra 8 GB and newer memory bandwidth make it the current consumer flagship for AI video. Preview iteration with the distilled pipeline runs comfortably at higher resolution.
48 GB tier — L40S, RTX 6000 Ada
Forty-eight-gigabyte workstation cards move you into LoRA fine-tuning territory. Standard LoRA training runs at reasonable batch sizes on a single card, IC-LoRA training becomes practical (doubled sequence length compared to plain LoRA), and inference workloads no longer need FP8 to fit.
This is the tier where you can leave a training run going overnight while continuing to iterate on inference during the day.
80 GB tier — A100 80GB, H100
An NVIDIA H100 or A100 80GB is where full model fine-tuning becomes tractable and multi-GPU training scales cleanly. The distilled model checkpoint can be trained and audio-video LoRA training preserves synchronization at full sequence length.
The H100's Hopper architecture also gets the largest speedup from fp8-scaled-mm via TensorRT-LLM.
What VRAM tier does your LTX-2.3 job need?
LTX-2.3 specifically ships with two checkpoint variants: ltx-2.3-22b-dev.safetensors and ltx-2.3-22b-distilled.safetensors. Both are 22B-parameter models; the distilled variant is what the DistilledPipeline loads for its 8-sigma fast path.
Two-stage pipelines also require the spatial upscaler checkpoint (ltx-2.3-spatial-upscaler-x2-1.0.safetensors or the x1.5 variant) and the distilled LoRA checkpoint. All of these must fit in GPU memory at once during inference.
Concrete VRAM planning: for preview and prompt iteration on the distilled checkpoint with FP8 cast and expandable segments, plan on 16 GB minimum. For production two-stage at 720p–1080p target resolution, 24 GB is comfortable, 32 GB is comfortable with headroom for concurrent IC-LoRA.
For LoRA fine-tuning on the low-VRAM variant, 32 GB is the documented floor with INT8 quantization. For full model training or audio-video LoRA at full sequence length, 80 GB+ is the target.
What if my GPU doesn't clear the bar?
If local hardware doesn't reach the tier your workload needs, the hosted LTX-2 API at docs.ltx.video runs the same generation stack without any local GPU. This is the pragmatic on-ramp for teams that want to prototype at scale before committing to workstation hardware, or for production pipelines that need burst capacity beyond what a single 4090 or 5090 delivers.
Note that the hosted API and the open-source Python pipelines have different parameter surfaces — cfg_scale, stg_scale, and rescale_scale are open-source Python parameters, not hosted API knobs — so recipe transfers between surfaces need explicit mapping.
Beyond the GPU: what else matters
Supporting hardware trips up more setups than people expect. CPU and RAM matter for preprocessing reference videos in IC-LoRA workflows (the full reference video must be loaded and structural signals extracted before generation starts).
NVMe throughput matters when the checkpoint, upscaler, distilled LoRA, and Gemma text encoder all load into memory at pipeline startup — a slow SSD adds noticeable seconds per run. Cooling and thermal headroom matter for long training runs, where sustained wattage over hours will throttle any card that can't shed the heat. Power supply sizing must cover the card plus the CPU under sustained load.
Attention optimizations that stretch every card
Two software optimizations pull real performance out of any tier. xFormers (installed with uv sync --extra xformers) reduces memory pressure in attention operations across all supported GPUs. Flash Attention 3 is the biggest lever on Hopper GPUs specifically.
Both are drop-in and worth enabling before spending on the next hardware tier.
Cost per generation: the math worth doing
Zero marginal cost is the argument for running locally. A production two-stage 1080p generation on a 24 GB card is measured in electricity and time — no per-second billing. A hosted API call at scale is measured in dollars and reliability.
The break-even calculation depends on batch volume. A team doing dozens of iterations per day per creator hits break-even on a workstation GPU inside a quarter; a team running a single client project a month rarely does. Do the math with your actual generation count before assuming hardware is the answer.
How to decide: a three-step framework
Pick the workload you spend the most time on, not the one that impresses. Preview iteration, production generation, LoRA training, or full fine-tuning — one of these gets 80% of your GPU hours.
Optimize for that. Cross-check against the VRAM floors above for LTX-2.3 specifically, with FP8 quantization enabled. If your primary workload fits with FP8 headroom, buy that tier. If you keep bumping the ceiling, jump a tier — the alternative is fighting OOM errors every session, and that cost compounds fast.
Summary
LTX-2.3 runs comfortably on consumer GPUs when the workload matches the card. Preview iteration on 16 GB cards is viable with the DistilledPipeline and FP8 quantization. Production two-stage generation is the sweet spot for RTX 4090 (24 GB) and RTX 5090 (32 GB).
LoRA fine-tuning needs 48 GB workstation cards for practical batch sizes. Full model training and audio-video LoRA at full sequence length target 80 GB H100 or A100 hardware.
FP8 quantization via fp8-cast or fp8-scaled-mm pulls the VRAM floor down roughly one card generation, and gradient estimation reduces inference steps from 40 to 20–30 while maintaining quality. When local hardware can't reach the tier your workload needs, the hosted LTX-2 API at docs.ltx.video is the on-ramp.
Start with the workload, match to the tier, then pick the card that fits your VRAM ceiling with FP8 headroom to spare.
Ready to run LTX-2.3?
Clone the LTX-2 repository, download the checkpoints from HuggingFace, and get the pipelines running locally in an afternoon. Or start with the hosted LTX-2 API if you want to skip the hardware question entirely.
