LumiVid generates HDR video from a single SDR input via ACEScct latent alignment and dual-canvas LoRA — no masks, text prompts, or reference images.
LumiVid generates HDR video from a single SDR input via ACEScct latent alignment and dual-canvas LoRA — no masks, text prompts, or reference images.
High dynamic range (HDR) video captures continuous scene radiance essential for professional color grading and post-production workflows, yet generative video diffusion models remain bound to 8-bit standard dynamic range (SDR) formats. Existing adaptation methods either alter the VAE latent space or rely on auxiliary inputs like luminance masks, text prompts, or external reference images. However, relying on auxiliary inputs introduces manual overhead, makes the exact prompt hard to guess, and yields less stable results that often fail to remain consistent with the original footage.
We present LumiVid, a framework that generates HDR video directly from a single SDR input, without requiring masks, text prompts, or external reference images. Key to our method is mapping the HDR video into a logarithmic color space representation, aligning high-magnitude radiance with the existing latent space of a frozen video VAE without losing luminance details. Fine-tuning a dual-canvas LoRA adapter on just 300 synthetic paired clips enables data-efficient SDR-to-HDR translation while preserving temporal consistency. To prevent shortcut pixel copying during training and force the model to infer missing radiance and details from its learned priors, we introduce camera-mimicking condition degradations on the SDR reference input. Extensive evaluations show that LumiVid outperforms state-of-the-art baselines by +5.5 dB PU21-PSNR on cinema-grade ARRI footage, producing high-fidelity half-float EXR video with exceptional re-exposure latitude across extreme exposure pulls.
| Image | Video | ||||
| SDR (Input) | ![]() | ![]() | ![]() | ![]() | |
| HDR (Ours) | ![]() | ![]() | ![]() | ![]() | |
-2 and a sunset still at EV 0. The video panel shows two times from a rooftop clip at EV -1 and EV -3.High dynamic range (HDR) imagery captures scene radiance over a wide range of intensities, enabling faithful representation of highlights, shadows, and subtle illumination effects that are lost in standard imagery. This capability is essential for realistic rendering, post-production, professional color grading, and emerging display technologies.
Professional film and visual effects workflows rely on scene-referred data: a format where pixel values represent the actual physical light captured by the camera sensor. In this system, diffuse white (the light bouncing off a flat, non-shiny surface like a white sheet of paper) is set to a baseline value of 1.0. Brighter light isn’t capped at 1.0. Specular glare, car headlights, and direct sunlight extend unbounded into values of 10, 100, or even thousands. Holding onto real-world light values makes post-production reversible. Colorists adjust exposure in stops (the standard measurement of camera exposure, where each stop doubles or halves the light energy (Pixelnew=Pixel× 2stops). Since the video file retains data far above 1.0, dropping the exposure by a few stops reveals crisp details inside a glowing window instead of leaving a blown-out white blob. Shadow recovery also avoids color banding because dark tones were saved as continuous light levels. This linear math is also essential for realistically blending 3D CGI with live-action footage. Only at the end of the workflow is the footage converted into display-referred data, a set of image instructions tailored specifically to the physical limits of a monitor. For standard SDR screens, this meant squashing all light into a rigid 8-bit scale from 0 to 255, where 255 is the absolute brightest white the screen can produce. Modern HDR screens can display 10-bit or 12-bit color depths, but even they require a final display transform to map unbounded light values onto what the screen hardware can physically output.
This compression explains why video AI models struggle to generate native HDR. Video AI models are trained mostly on web videos stored in 8-bit SDR formats, where a piece of white paper and the sun are both squashed into the exact same ceiling of 255. These models learn exclusively from flattened, monitor-bound data, they lack an internal knowledge of how real physical light behaves beyond standard display limits.
A natural strategy for extending diffusion models to HDR is to leverage the visual structure, semantics, and temporal dynamics already learned by pretrained video models [19]. One common approach is to train a dedicated HDR encoder or fine-tune VAE decoders to emit linear radiance [41], creating an HDR-specific latent space. However, altering or replacing the encoder shifts the underlying latent manifold, forcing the downstream diffusion model to adapt to an unfamiliar space. This dependency demands massive HDR datasets and makes training complex and computationally expensive.
Drastically altering the latent space breaks compatibility with the model’s pretrained features. Keeping the HDR representation aligned with the model’s native latent space allows for data-efficient adaptation while preserving existing capabilities, such as video editing, inpainting, and controlled generation, directly in the HDR domain without needing to retrain those tools.
DiffHDR [51] has begun to explore aligning HDR with the model’s native latent space, but relies on external prompts and masks to synthesize missing highlights. In contrast, our method performs direct SDR-to-HDR translation conditioned only on the input video itself. We map scene-linear HDR into ACEScct, a standardized cinematic logarithmic color space (see Appendix B for color space fundamentals). Encoding this log video with the frozen SDR VAE yields latents already aligned with the pretrained manifold, allowing the generator to emit log-encoded HDR with lightweight fine-tuning and without a new encoder.
We train an in-context LoRA on a dual-canvas architecture, where SDR tokens condition the denoising of ACEScct HDR target tokens. 300 paired clips are sufficient: the LoRA learns a targeted dynamic-range conversion. To encourage the model to learn true HDR generation from an SDR source, we make the task even more challenging. Since SDR video already lacks detail in extreme luminance regions, we apply camera-mimicking degradations, such as aggressive clipping, heavy compression, and selective blur, to strip away even more information from the highlights and shadows. Combined with varied exposure shifts during training, this forces the network to rely on its internal priors to actively synthesize continuous, high-magnitude radiance rather than copying flat SDR pixels, eliminating the need for text prompts or luminance masks.
We evaluate on professional ARRI cinema footage and on 66 YouTube in-the-wild SDR videos. Both evaluation benchmarks comprise real cinematic footage featuring complex actor motion, dynamic lighting transitions, and non-rigid moving objects. Our findings show that HDR capabilities can be enabled in existing video models with high data efficiency, without architectural redesign or full retraining.
Our key contributions are:
∼300 synthetic paired clips while preserving temporal consistency. Despite training exclusively on synthetic camera orbits, our model generalizes seamlessly to real-world footage containing dynamic actor motion and complex scene transitions at inference.[0,0.55] on the SDR reference input vs. [0,1.0] on the HDR target), to prevent shortcut pixel copying during training, forcing the model to infer extreme high-magnitude highlights and deep shadow details from its learned priors. Video results, exposure pulls, and an overview are at https://hdr-lumivid.github.io/.The challenge of capturing and manipulating high dynamic range content has long been a focal point of computational photography. Early foundations were built on multi-exposure HDR recovery [12] and photographic tone mapping for display [35]. A landmark in forward tone-mapping (HDR-to-SDR) was the work of Fattal et al. [15], who pioneered gradient-domain dynamic range compression. By selectively attenuating large luminance gradients (global contrast) while preserving small gradients (local detail), they demonstrated that integrating the modified field via a Poisson equation could compress dynamic range without introducing edge artifacts. This was further refined by Farbman et al. [14], who introduced edge-preserving multiscale decompositions based on Weighted Least Squares (WLS) optimization, providing a robust framework for detail manipulation that effectively eliminated the halo artifacts common in non-linear filtering.
While these classical operators focused on compressing scene-linear HDR for bounded displays, the complementary task of Inverse Tone Mapping (ITM) aims to expand SDR content back into unbounded radiance. ITM evolved from early heuristic expansion operators [5, 36] to modern deep CNN-based reconstruction architectures that reverse the camera pipeline or estimate linear radiance from single exposures [13, 32, 27]. A significant step toward modern generative approaches was taken by Marnerides et al. [31], whose work on Deep HDR Estimation moved beyond deterministic pixel mapping by utilizing generative detail hallucination for saturated regions. More recent architectures improve efficiency via frequency-domain processing [45] or leverage specialized sensing modalities such as event cameras [48].
A parallel line converts SDRTV to display-referred HDRTV (PQ, BT.2020) rather than scene-linear radiance. HDRTVNet [11] introduced a practical SDR-to-HDRTV pipeline and the HDRTV1K benchmark; HDRTVNet++ [10] extends it with adaptive global color mapping, local enhancement, and highlight refinement. HDRTVDM [18] adds degradation models and HDRTV4K. We use HDRTVNet++ as a per-frame baseline, but it targets 10-bit HDR display delivery rather than 16-bit scene-linear EXR files.
Extending to video, prior work focuses on temporal alignment and consistency using optical flow [23], coarse-to-fine alignment [9], or specialized sensing modalities [48]. Recent methods further address robustness to illumination changes [50] and real-time performance [46], supported by large-scale HDR video datasets [37].
These approaches treat SDR-to-HDR as a reconstruction problem, learning dedicated mappings under restricted imaging assumptions. In contrast, we treat HDR generation through the lens of distribution alignment: pretrained video diffusion models already capture rich priors over illumination, highlights, and shadows. The key challenge is to expose and unlock this knowledge by aligning HDR input/output distributions with the latent representations expected by the frozen model.
Recent research has shifted toward leveraging generative models to synthesize missing radiance. Early works explored unsupervised HDR synthesis via GlowGAN, a StyleGAN variant [40] or applied conditional diffusion to multi-exposure image deghosting [47] and multi-exposure reconstruction [17]. Frame-level frameworks like BracketDiffusion [6] employ exposure-aware modeling to ensure consistency across simulated brackets, while other approaches adapt pretrained diffusion backbones through gain-map decomposition [16, 26] or zero-shot transfer [53]. LEDiff [41] enables text-to-HDR and LDR-to-HDR via exposure-aware latent fusion.
X2HDR [44] is concurrent image work. It adapts pretrained image diffusion models to HDR by operating in PU21 encoded perceptual space with a frozen VAE and LoRA fine-tuning—reflecting the view that HDR can be handled by mapping it into a compatible latent space. However, these methods rely primarily on display-oriented encodings and are restricted to single images. When applied frame-by-frame to video sequences, single-image diffusion models exhibit temporal flickering and lack cross-frame motion coherence.
Video diffusion models have rapidly advanced from temporal extensions of image diffusion [21] to large-scale generative foundation models such as Stable Video Diffusion [8], CogVideoX [49], and Wan2.1 [43]. Modern architectures, including LTX-Video [20] and LTX-2 [19], combine highly compressed spatial-temporal video VAEs with scalable Diffusion Transformers (DiT) [33] and rotary position embeddings [38]. Low-Rank Adaptation (LoRA) [22] has become the standard mechanism for parameter-efficient adaptation of these backbones, as demonstrated by concurrent lighting estimation frameworks like LuxDiT [25].
Two recent works explore video diffusion backbones for SDR-to-HDR conversion. DiffHDR (Yu et al., 2026) [51] shares our use of a frozen VAE, log-space encoding, and LoRA adaptation on a video DiT. However, it formulates SDR-to-HDR as a masked radiance inpainting task where luminance masks mark clipped regions, and external text prompts or reference images specify what content to synthesize. The other one is Tedla et al. (2026) [39] Fully fine-tunes Wan2.2 to generate a linearized three-exposure SDR bracket (-4/0/+4 EV) via temporal latent concatenation, subsequently fusing the bracket with a learned per-frame merger network. All VAE operations remain in native SDR space, forming HDR only post-decoding via classical bracket merging.
Our framework, LumiVid, differs from both approaches. Unlike DiffHDR, LumiVid treats conversion as SDR to HDR process. The input SDR video is the only input signal, requiring no masks, text prompts, or reference images. Unlike Tedla et al., who keep generative priors in SDR and rely on multi-exposure bracket generation, we map scene-linear HDR directly into ACEScct log space. This allows a frozen VAE to represent high-magnitude radiance directly and enables a lightweight LoRA to emit 16-bit scene-linear HDR video in a single pass, without requiring an exposure ladder or secondary fusion network.

Standard video diffusion models are bound to standard dynamic range (SDR) formats. We address SDR-to-HDR video generation by fine-tuning a Dual Canvas Lora adapter [7] on top of a frozen video diffusion model. The pretrained video that we are using is video DiT LTX-2.5 [19]. Given an SDR source video, the network generates a spatially and temporally aligned HDR target. During the training, the model gets clean SDR latents and noisy HDR target to denoise as a dual canvas . At inference, it receives the input SDR latents alongside Gaussian noise, iteratively denoising the noise into the final HDR video.
Our training recipe consists of three core components (Figure 2):
∼ 300 paired clips to learn targeted dynamic-range translation.To adapt a pretrained video diffusion model to HDR with lightweight fine-tuning, the model must operate on a latent distribution that closely matches what it was originally trained on. Pretrained video backbones contain learned priors about real-world physics, lighting, and motion acquired from web-scale SDR datasets. Since HDR video data is scarce, retraining the backbone or learning a new latent space would destroy these pretrained representations. Keeping the VAE encoder and decoder frozen ensures that we preserve the original model’s knowledge, enabling data-efficient adaptation.
However, directly encoding scene-linear HDR data with the frozen VAE fails. Unbounded, heavy-tailed RGB values deviate drastically from the VAE’s expected SDR input range, causing severe reconstruction artifacts. As shown in Table 1, measuring the direct VAE roundtrip (x→Encoder→Decoder→x) reconstruction error on linear RGB frames yields poor quality, resulting in a low PU21-PSNR of 13.99 dB and a high L2 error of 2.81. This low performance is expected since the VAE was trained on SDR data where the color range was 0-1, and not unbounded. A high roundtrip reconstruction error indicates that the VAE latents fail to faithfully capture the HDR video; because radiance details are lost during encoding, the decoder cannot accurately reconstruct the original scene. These degraded latents cannot serve as a viable target for diffusion training.
To bridge this representation gap while maintaining our frozen latent space, we map scene-linear HDR into ACEScct, a standardized cinematic logarithmic color space. ACEScct compresses unbounded radiance into a quasi-logarithmic scale aligned with the input statistics expected by the frozen VAE. Evaluating the VAE roundtrip reconstruction through the full ACEScct HDR target space (clamped at 1.0) yields an improvement, boosting the PU21-PSNR to 29.97 dB and reducing the L2 error to 0.69. By contrast, clamping the SDR reference condition at 0.55 (diffuse white) yields 23.02 dB PU21-PSNR, explicitly stripping highlights above white so the reference cannot leak high dynamic range codes into the condition canvas. We also show that clipping the ACEScct at 1.0 and then converting back to ACEScg results in almost no loss in the data with PU21-PSNR > 99.0
This confirms that the frozen SDR VAE already possesses sufficient capacity to encode and decode HDR content once the color distribution is properly aligned. By maintaining this alignment, we can teach the model to generate full HDR video while retaining the rich generative priors of the original model.

zref. These reference latents are concatenated as a dual canvas with noise and iteratively denoised by the Video DiT with LoRA adapters. The output latents are VAE-decoded and inverted through ACEScct to recover ACEScg scene-linear HDR video. The VAE and DiT remain frozen throughout; only the lightweight LoRA adapters were trained via flow matching.| Input | Poly Haven | ARRI | ||
|---|---|---|---|---|
PU21-PSNR ↑ | L2 ↓ | PU21-PSNR ↑ | L2 ↓ | |
| Linear unclipped | 13.99 | 2.81 | 14.77 | 0.88 |
ACEScct 0.55 | 23.02 | 2.80 | 24.15 | 0.80 |
ACEScct 1.0 | 29.97 | 0.69 | 30.17 | 0.16 |
ACEScct 1.0, no VAE | >99.00 | ∼ 0 | >99.00 | ∼ 0 |
Logarithmic Formulation Let x denote scene-linear radiance in ACEScg. We map x into the normalized ACEScct logarithmic space via the piecewise transformation T(x):
T(x)=10.540· x+0.073x<2-7,log2(x)+9.7217.52x≥ 2-7.(1)
The linear region below x=2-7 avoids logarithmic encoding and stabilizes numerical representations in deep shadows. Following iterative denoising and VAE decoding, scene-linear radiance is reconstructed via the inverse transformation T-1(y) for an encoded log value y∈[0,1]:
T-1(y)=y-0.07310.540y<0.155,217.52· y-9.72y≥ 0.155.(2)
(Note: Constants above are rounded for readability; exact double-precision coefficients follow the standard ACEScct specification. [2])
We adapt a frozen LTX-2.5 video Diffusion Transformer (DiT) [19] using a rank-128 dual-canvas LoRA [7] trained with flow matching, keeping the VAE frozen. Following the dual-canvas formulation, the input sequence is constructed by concatenating two latent streams along the sequence dimension: a reference canvas containing clean (un-noised) SDR latents, and a target canvas containing noisy HDR latents. During denoising, the target HDR tokens directly attend to the clean SDR tokens through the DiT’s self-attention mechanism. To maintain precise spatial-temporal correspondence, the 3D Rotary Position Embeddings (RoPE) of the SDR reference tokens are aligned directly with those of the target HDR tokens. The flow-matching diffusion loss is computed only on the target HDR tokens.
This in-context conditioning allows the LoRA adapter to learn a localized dynamic-range mapping from the ACEScct SDR reference to the ACEScct HDR target. The frozen backbone already retains priors over video motion and physics, thus, the model does not need to learn video synthesis from scratch, enabling rapid convergence on just a few hundred paired clips (∼ 300). Text-to-HDR generation drops the SDR reference canvas and is detailed in Appendix J.
High-quality HDR video data is scarce in the wild, thus, we construct a synthetic paired SDR-HDR video dataset by rendering synthetic camera orbits through high-resolution PolyHaven 360° HDRi environment maps [34]. We produced approximately 300 49-frame scene-linear HDR video clips. Each clip is converted to ACEScg and mapped into ACEScct log space via T(x).
While our training dataset uses camera orbits through static HDRi environments, all evaluation benchmarks (ARRI cinema footage and youtube in-the-wild clips) are conducted strictly on real-world video footage featuring dynamic real motion, including actor movements, non-rigid object dynamics, complex occlusions, and cinematic camera work, rather than synthetic camera orbits. Generalization from synthetic training data to real-world footage succeeds because motion and temporal priors remain in the frozen DiT backbone. The backbone handles scene dynamics and physics, while the LoRA adapter focuses exclusively on translating ACEScct SDR codes to ACEScct HDR codes. Training the LoRA on PolyHaven HDRI camera orbits provides rich photometric highlight and shadow variations while relying on the frozen backbone for temporal and motion dynamics.
Given the HDR rendered videos we can derive the corresponding SDR conditioning sequences used during training. In ACEScct log space, diffuse white (x=1.0) maps to T(1.0)≈ 0.55, while the upper boundary y=1.0 caps extreme scene radiance at T-1(1.0)≈ 223× diffuse white. To force the model to learn true dynamic-range translation from the SDR reference to the HDR target we use an asymmetric clamping strategy during training.
[0,0.55]. This strips highlight radiance above diffuse white from the reference input, ensuring the SDR condition carries no latent HDR codes.[0,1.0], exposing target latents to the full dynamic range up to ≈ 223× white.This 0.55 vs. 1.0 split forces the network to infer extreme high-magnitude highlights (y∈(0.55,1.0]) from learned diffusion priors rather than leaking pixels from the conditioning canvas.
To encourage the model to synthesize true HDR and make the SDR-to-HDR training task more challenging, we apply three camera-mimicking degradation operators to the SDR reference canvas prior to VAE encoding:
y≈ 0.55), Gaussian blur simulates optical dispersion and lens bloom. To prevent the model from overfitting to a fixed blur pattern, the Gaussian standard deviation σ∈[5,12] and kernel size are dynamically randomized across clips. For crushed shadows near the log toe (y≈ 0.073), an area-downsampling and cubic-upsampling roundtrip strips fine spatial texture.y=0.5+(y-0.5)· f. Anchoring around a fixed log midpoint (0.5) rather than a per-channel frame mean prevents white-balance and color-neutral drift while driving highlight and shadow values into clipping boundaries.CRF∈[23,35]), introducing macroblocking and quantization noise directly into the SDR input.Full hyperparameter distributions and implementation specifics are provided in Appendix E.
In addition to structural degradations, we apply joint exposure shifts across both the SDR condition and HDR target canvases. Simultaneously shifting both streams by shared random exposure offsets (spanning -2,0,2 EV stops) exposes the network to diverse luminance scales. This forces the model to recover both extreme highlights and deep shadow details across varied exposure levels while preserving spatial alignment between condition and target.
Together, these operations erase structural details in clamped or blurred regions, forcing the network to synthesize continuous physical radiance from its learned diffusion priors rather than reproducing input pixels. Consequently, the model generates smooth, unclipped specular gradients when pulling exposure down (EV-4/-3) and reveals coherent spatial texture when pushing shadows up (EV+3/+4).
During inference, an input SDR video is converted from Rec.709 primaries to linear ACEScg, mapped via T(x) into the clamped ACEScct [0,0.55] and encoded by the frozen VAE. The Video Diffusion Transformer (DiT) with dual-canvas LoRA adapters performs iterative denoising in ACEScct latent space. Finally, decoded latents in ACEScct [0,1] are mapped back to scene-linear ACEScg via T-1(y) and saved as 16-bit half-float EXR files.
| Input (SDR) | ![]() | ![]() | ![]() |
| HDRTVNet++ | ![]() | ![]() | ![]() |
| X2HDR | ![]() | ![]() | ![]() |
| EV -7 | EV -3 | EV 0 |
| DiffHDR | ![]() | ![]() | ![]() |
| Tedla et al. | ![]() | ![]() | ![]() |
| LumiVid | ![]() | ![]() | ![]() |
| EV -7 | EV -3 | EV 0 |
-7, -3, and 0. Insets show the right headlight. At EV 0 the opening is blown out in every row; pulling exposure down shows what each method stored in the highlights. LumiVid recovers cloud structure through the door and a structured headlight. HDRTVNet++ barely expands the range. DiffHDR stays close to the SDR input (Table 3: Y99.9=0.84). Tedla et al. [39] (SDR2HDR) leaves little structure in the lamp at EV -7. X2HDR recovers some sky but clips the lamps.| Input (SDR) | ![]() | ![]() | ![]() | ![]() |
| DiffHDR | ![]() | ![]() | ![]() | ![]() |
| Tedla et al. | ![]() | ![]() | ![]() | ![]() |
| LumiVid | ![]() | ![]() | ![]() | ![]() |
| EV − 4 -4 | EV − 2 -2 | EV 0 0 | EV + 2 +2 |
-4, -2, 0, and +2. EV 0 matches the SDR look; negative EVs show recovered highlights and EV +2 lifts the shadows. LumiVid keeps a compact solar disk and the cloud structure around it. DiffHDR stays close to the SDR input (Table 3: Y99.9=0.84). Tedla et al. leaves a small pale disk at EV -4.| EV -3 | EV 0 | EV +3 | EV -3 | EV 0 | EV +3 |
![]() | ![]() | ![]() | ![]() | ![]() | ![]() |
| EV -5 | EV -3 | EV 0 | EV -5 | EV -3 | EV 0 |
![]() | ![]() | ![]() | ![]() | ![]() | ![]() |
| EV -4 | EV 0 | EV +4 | EV -4 | EV 0 | EV +4 |
![]() | ![]() | ![]() | ![]() | ![]() | ![]() |
| SDR (Input) | HDR | ||||
Datasets. We evaluate LumiVid on two benchmarks. ARRI Cinema Benchmark [4] which consist of 50 professional cinema clips with authentic 12-bit HDR ground truth. Input SDR videos are generated by forward tone-mapping the raw HDR plates to 8-bit Rec.709. These clips contain real-world cinematography, including motion, and dynamic lighting transitions. The second benchmark is In-the-Wild Benchmark: 66 diverse SDR video clips collected from YouTube. This benchmark tests generalization on real-world non-rigid motion (e.g., dynamic objects, human faces and motion, lighting transitions) without HDR ground truth.
We compare LumiVid against four state-of-the-art SDR-to-HDR baselines spanning both per-frame image models and native video diffusion backbones.
All methods run at 720× 1280 resolution. We run 49-frame on ARRI and 17-frame on in-the-wild clips, outputting Rec.709 scene-linear EXR files. Tedla et al., works only on 17 frames, thus when running inference on ARRI benchmark, we run three 17-frame windows, that are tiled at starting frame indices 0, 16, and 32 (overwriting 1-frame overlaps) to cover 49 frames.
Evaluation Metrics. All generated EXR files are mapped to Rec.709 scene-linear space prior to metric calculation to ensure strict photometric alignment with the other baselines. We evaluate our method using common hdr video metrics, across four axes:
0–10 scale.yt and yt+1). We also report t-LPIPS [52] - LPIPS between consecutive PU21 frames. Higher F2F-PSNR and lower t-LPIPS indicate stable, flicker-free video generation.EV-3 to inspect recovered highlight detail and EV+3 for shadow detail) followed by sRGB tone-mapping.Y (where Y=1.0 represents diffuse white). In a scene-referred frame only a very small part of the pixels should reach extreme high values, a light, the sun, a specular, a sparkle, while the rest of the frame sits between 0-1. We therefore read the tail with a high percentile rather than a statistic of the whole frame. We define the dynamic range in stops as: Stops=log2(Y99.9) (3) where Y99.9 is the 99.9th percentile of Y over all pixels in a clip. The percentile is high enough to fall on that small highlight set. We report the median over the 66 clips.Flicker, highlight recovery, and video quality are best judged in the supplementary video and in a pairwise user study vs. HDRTVNet++, X2HDR and DiffHDR (§4.3).
Table 2 reports ARRI GT-referenced fidelity and temporal stability. LumiVid leads in PU21-PSNR, ColorVideoVDP, and F2F. While HDRTVNet++ achieves equal or slightly better results on t-LPIPS, PU21-SSIM, its PU21-PSNR is much worse, indicating that its output fails to achieve true HDR dynamic range. Qualitative crops at EV-3 (highlights) and EV+4 (shadows) are shown in Figure 6. In-the-wild baselines are presented in Figures 4 and 5.
| Method | PU21-PSNR ↑ | PU21-SSIM ↑ | ColorVideoVDP ↑ | F2F ↑ | t-LPIPS ↓ |
|---|---|---|---|---|---|
| HDRTVNet++ | 22.43 | 0.935 | 7.29 | 44.81 | 0.0057 |
| X2HDR | 18.27 | 0.818 | 3.19 | 44.60 | 0.0143 |
| DiffHDR | 24.28 | 0.743 | 4.57 | 40.36 | 0.0075 |
| Tedla et al. [39] | 16.80 | 0.687 | 4.77 | 43.59 | 0.0070 |
| LumiVid | 29.79 | 0.915 | 7.92 | 54.28 | 0.0059 |
Table 3 scores the in-the-wild benchmark. F2F / t-LPIPS for native video; MUSIQ and CLIP-IQA+ at EV -3 / +3 for highlights and shadows. LumiVid leads those columns.
LumiVid sits +2.8 stops above white at Y99.9, (6.7×) much higher than the other baselines. HDRTVNet++ is a mild display lift (1.7×). Flicker, highlight recovery, and the exposure pulls are on the project page. The comparisons are best viewed on an HDR display.
| Quality | Range | ||||||
|---|---|---|---|---|---|---|---|
| Method | F2F ↑ | t-LPIPS ↓ | MUSIQ-3 ↑ | CLIP-3 ↑ | CLIP+3 ↑ | Y99.9 ↑ | Stops ↑ |
| HDRTVNet++ | 37.72 | 0.065 | 43.6 | 0.412 | 0.355 | 1.66 | +0.73 |
| X2HDR | 38.19 | 0.068 | 44.4 | 0.426 | 0.352 | 2.06 | +1.04 |
| DiffHDR | 38.84 | 0.065 | 40.8 | 0.425 | 0.385 | 0.84 | -0.26 |
| Tedla et al. [39] | 36.55 | 0.062 | 38.2 | 0.403 | 0.383 | 0.73 | -0.45 |
| LumiVid | 45.15 | 0.059 | 46.3 | 0.434 | 0.417 | 6.74 | +2.75 |
Full-reference metrics like PU21-PSNR and ColorVideoVDP do not fully capture subjective visual quality or subtle temporal flickering. Thus, we conducted a pairwise 2AFC (Two-Alternative Forced Choice) perceptual user study comparing LumiVid against DiffHDR, X2HDR, and HDRTVNet++. A total of twenty one raters evaluated 30 video pairs each (10 scene pairs per baseline). The study evaluated three key dimensions: (1) temporal stability, color preservation, and flicker; (2) spatial detail and highlight texture recovery; and (3) overall visual quality. The EXR files were displayed as hlg files in three exposures, -3, 0, 3. To ensure good visual evaluation, all ratings were collected on a 1000-nit MacBook Pro display operating in native HDR mode. Figure 7 is flicker, detail, and overall quality. On this HDR display, raters prefer LumiVid to every baseline.
We conduct ablation experiments to isolate two design choices in our framework: (1) whether the generative diffusion prior of the Video DiT is necessary for high quality HDR recovery, or if a fine-tuned VAE decoder is sufficient; and (2) whether camera-mimicking condition degradations are required to prevent shortcut copying of 8-bit SDR pixel structures.
Isolating the Video DiT Backbone (VAE Decoder Ablation): A fundamental question is whether SDR-to-HDR video translation requires a generative diffusion process, or if it can be solved as a lightweight latent-to-pixel decoding task. Training a standalone encoder-decoder network from scratch would discard the rich representations learned by the pretrained VAE model and demand large-scale HDR video datasets that do not exist. Operating directly in the VAE’s latent space preserves these representations.
Furthermore, as established in §3.1, mapping scene-linear HDR into ACEScct allows the frozen VAE to represent log radiance without altering the latent manifold expected by the Video DiT (Table 1). This raises a compelling question: if the frozen SDR VAE encoder already produces latents that map well to ACEScct, can we simply fine-tune a VAE decoder to decode those latents directly into HDR frames in a single feed-forward pass?
To test this, we construct a VAE decoder-only baseline. We fine-tune the pixel decoder directly on ACEScct targets while keeping the VAE encoder frozen, completely omitting the Video DiT. At inference, SDR video is encoded and decoded to HDR in a single pass. This isolates whether the Video DiT is required for temporal consistency and high-magnitude radiance synthesis, or if log encoding plus a fine-tuned pixel decoder is sufficient.
| Input (SDR) | ![]() | ![]() | ![]() | ![]() |
| VAE decoder | ![]() | ![]() | ![]() | ![]() |
| LumiVid | ![]() | ![]() | ![]() | ![]() |
0). The VAE decoder can lift range in the window, sky, and sun; but it does not put structure there. Faces, cloth, and foliage smooth, and those highlights clip. LumiVid is adding the HDR details and structure on top of the SDR input. Best to view these results in the attached project page.| Split | Metric | VAE Decoder | LumiVid (Main) |
|---|---|---|---|
ARRI (n=50) | PU21-PSNR ↑ | 29.43 | 29.79 |
ColorVideoVDP ↑ | 7.60 | 7.92 | |
F2F ↑ | 46.35 | 54.28 | |
t-LPIPS ↓ | 0.0082 | 0.0059 | |
In-the-Wild (n=66) | F2F ↑ | 43.33 | 45.15 |
t-LPIPS ↓ | 0.062 | 0.059 | |
MUSIQ-3 | 43.2 | 46.3 | |
CLIP-IQA+-3 | 0.407 | 0.434 | |
CLIP-IQA++3 | 0.380 | 0.417 |
Fine-tuning only the VAE decoder for SDR-to-HDR translation fails to deliver high-fidelity, temporally stable HDR video. While a spatial VAE decoder can predictably re-map diffuse mid-tones, it lacks the generative capacity of the Video DiT backbone to synthesize unclipped radiance or enforce cross-frame temporal coherence. As detailed in Table 4, the limitations of this decoder-only baseline are evident across both ground-truth-referenced and in-the-wild evaluations.
Under a 3-stop highlight pull (EV-3), the VAE decoder fails to synthesize plausible structure in overexposed regions, producing flat, featureless clipping plateaus. This is directly reflected in in-the-wild no-reference metrics, where MUSIQ-3 drops from 46.3 (LumiVid) to 43.2 and CLIP-IQA+-3 falls from 0.434 to 0.407. Symmetrically, pushing shadow regions up by 3 stops (EV+3) causes CLIP-IQA++3 to drop from 0.417 to 0.380, demonstrating the decoder’s inability to reconstruct details in crushed darks. The VAE decoder exhibits significant temporal instability. On the ARRI benchmark, frame-to-frame fidelity (F2F) drops sharply from 54.28 to 46.35, while temporal flicker (t-LPIPS) worsens from 0.0059 to 0.0082. Qualitatively (Figure 8), whereas LumiVid leverages diffusion priors to selectively generate extreme peak radiance dictated by scene semantics, the VAE decoder merely expands pixel values uniformly—causing sky regions to oversmooth, specular highlights to smear into halos, and fine geometric edges to vanish. These results confirm that temporally consistent dynamic-range expansion requires the full generative capacity of the Video DiT backbone.
Impact of Camera-Mimicking Degradations: To force the model to infer unclipped radiance rather than copy input pixels, we apply contrast clipping, compression, and highlight/shadow blur to the SDR conditioning canvas during training. Table 5 evaluates the impact of these degradations. Training with no augmentation lowers PU21-PSNR by nearly 1.0 dB (28.83 vs. 29.79 dB) and degrades ColorVideoVDP (7.45 vs. 7.92). Without contrast clipping, the SDR reference retains quantized highlight steps; the network learns a shortcut by simply stretching 8-bit steps into linear space, resulting in severe posterization and temporal flickering. Combining contrast clipping, highlight/shadow blur, and joint exposure shifts, forces the Video DiT to rely on its diffusion prior to synthesize continuous, unclipped radiance.
| Config | PU21-PSNR ↑ | ColorVideoVDP ↑ | F2F ↑ |
|---|---|---|---|
| Full augs (main) | 29.79 | 7.92 | 54.28 |
| No augmentation | 28.83 | 7.45 | 49.14 |
| Blur only (no contrast) | 29.11 | 7.39 | 49.95 |
We presented LumiVid, an SDR-to-HDR method that generates temporally coherent wide range HDR video from a single SDR input via a video diffusion model backbone. We showed that the barrier is not model capacity but how we align HDR content with pretrained models. A video diffusion transformer trained only on SDR data already possesses rich knowledge about light transport and dynamic range. Unlocking this knowledge requires solving two problems: mapping HDR into the frozen VAE’s learned distribution, and forcing the model to learn to reconstruct extreme-luminance regions from its learned priors. With these in place, a LoRA adapter adding less than 1% parameters, trained for only 5,000 steps on ∼300 clips (∼8 hours, single GPU), produces temporally coherent float16 HDR video that outperforms the strongest dedicated SDR-to-HDR baseline. Several extensions are natural. Scaling the training data to include real-world HDR video captures would strengthen generalization. Incorporating perceptual HDR metrics such as HDR-VDP-3 [30] or ColorVideoVDP [29] as training objectives could further improve perceptual quality.
Our logarithmic alignment preserves the backbone’s native latent space, exciting direction for future work is extending this framework to other pretrained video capabilities, such as HDR video editing, HDR inpainting, and fine-grained visual control.
We would like to thank Alon Yaar and Alon Alpert from the LTX R&D group, and Omer Rabinovoch, Michel Angielczyk, and Yoav Lutzky from the LTX Creative group, for their guidance and support throughout the development of LumiVid. We are especially grateful to Jean Dominique Vandenberg (JD) for sharing his expertise in color science.
The project page, including the overview video, is the intended way to judge the results: https://hdr-lumivid.github.io/
The human color gamut is the boundary of human color vision, drawn as the CIE 1931 chromaticity diagram (Figure 9). The triangles inside it, spanned by red, green, and blue primaries, are the smaller subsets that a given screen, camera, or printer can actually reproduce.
Gamut is which chromaticities exist at all. Most SDR video is stored as Rec.709: a small triangle inside the full locus. Saturated LEDs, neon, and many camera-native colors lie outside it and are clipped onto the Rec.709 sides. An HDR working space uses a wider triangle so those chromaticities remain distinct instead of collapsing to the Rec.709 primaries.
Quantization is whether those values are a continuum or a palette. Consumer SDR is 8-bit integer per channel after a gamma curve: 256 discrete codes. Nearby radiances become the same code; an exposure pull or a grade shows banding. HDR is stored as scene-linear floating point (or 10–12-bit PQ on a display curve).
Light strength is how bright the sample is relative to a white surface in the scene. SDR is display-referred and clipped to a peak, historically about 100 nits. The sun, a headlamp, and a white wall can share the same code. HDR is scene-referred: we take x=1 as diffuse white and let highlights be many times that.
Most videos on the web are Rec.709 since Rec.709 was specified for CRT-era television: D65 white, those three primaries, and a peak the display could actually make. Cameras, broadcast, discs, and later phones and web codecs (H.264/H.265 SDR, YouTube) copied that contract because the device at the end of the chain was a Rec.709 display with an 8-bit pipeline. HDR panels exist now, but not at a scale close to Rec.709 video. The pretrained video models we adapt were trained on that Rec.709 corpus, so their VAE expects Rec.709-like statistics.
Rec.709 is a display/camera encoding: D65, the small triangle in Figure 9. ACEScg is a scene-linear working space from the Academy Color Encoding System [1]: AP1 primaries -wider than Rec.709, inside AP0. (Rec.709 primaries lie inside AP1.), ACES white near D60, and linear RGB with x=1 equal to diffuse white and no upper bound. AP0 is the ACES2065-1 container gamut; its blue primary is imaginary (below the spectral locus), which is why the container can wrap Rec.709, AP1, and several camera encodings without clipping chromaticity. AP1 is the working gamut we actually compute in. ACEScg is not a display format: it is unbounded linear light on a wide triangle.

xy chromaticity. Rec.709 is the small SDR triangle (D65): the gamut of almost all web and broadcast video. ACEScg is AP1 (dashed gold), a wider working triangle inside the AP0 container (solid gold). AP0’s blue primary is imaginary (below the spectral locus), so the container can wrap Rec.709 and AP1 without clipping chromaticity. The diagram shows chromaticity only; light strength (how far above white) is not plotted.At inference we decode the SDR file with the sRGB EOTF and convert Rec.709 linear to ACEScg and then to ACEScct. The reverse path writes ACEScg EXR; metrics convert ACEScg to Rec.709 linear so every method is scored in the same space as the ARRI ground truth (Appendix F).
Linear ACEScg is a poor input to a frozen SDR VAE: the distribution is heavy-tailed and part of the pixels sits far above the [0,1] codes the VAE saw in training. ACEScct is a per-channel log of AP1.
Let x be an ACEScg linear RGB channel. The forward map is
T(x)=10.540237742x+0.072905534x≤ 0.0078125,log2(x)+9.7217.52x>0.0078125.(4)
The inverse is
T-1(y)=y-0.07290553410.540237742y≤ 0.15525114155,2y· 17.52-9.72y>0.15525114155.(5)
T(1)≈0.555 (diffuse white) and T-1(1)≈223. The linear toe below x=2-7 avoids log 0 in deep shadows.
Before the VAE we clamp ACEScct. On the SDR canvas the upper clamp is 0.55, just below T(1), so the condition cannot present highlights above white (T-1(0.55)≈0.94). On the HDR canvas the upper clamp is 1.0 (≈223× white).
The 0.55 vs. 1.0 split stops the LoRA from copying values that would otherwise sit in both ranges. Camera-mimicking degradations are applied on the SDR canvas after this clamp. ACEScct is a production log color space with a documented inverse.
Following AVControl [7], we construct an in-context dual-canvas framework by concatenating the SDR reference canvas with the HDR target canvas in the spatial-temporal sequence dimension. The SDR condition canvas is encoded once by the frozen VAE and injected as reference tokens, while the HDR canvas contains the noisy target latents. Noisy latents are not concatenated along the channel dimension. For text-to-HDR generation, the SDR condition canvas is simply dropped (Appendix J). Training the LoRA in this dual-canvas fashion is exceptionally data-efficient because the backbone does not need to learn generative video synthesis from scratch; it merely learns a targeted dynamic-range translation from the ACEScct SDR sub-space to the full ACEScct HDR range.
We train a rank-128 LoRA adapter on the frozen LTX-2.5 Video DiT backbone using flow matching for 5,000 steps on ∼300 paired 49-frame clips. The VAE encoder and decoder remain completely frozen throughout training. Optimization takes approximately eight hours on a single NVIDIA A100 GPU. The trainable LoRA parameters represent less than 1% of the total backbone parameters. At inference time, we use 8 denoising steps with standard flow matching sampling.
During inference, the input SDR video file is decoded using the standard sRGB Electro-Optical Transfer Function (EOTF) to obtain scene-linear values, converted from Rec.709 color primaries to ACEScg scene-linear space, mapped into the log ACEScct SDR range [0,0.55] via T(x), and encoded into latent space by the frozen VAE. The Video DiT with rank-128 LoRA adapters performs iterative denoising directly in ACEScct latent space. Following VAE decoding, output latents in ACEScct [0,1.0] are inverted back to scene-linear ACEScg via T-1(y) and exported as 16-bit half-float EXR files. For metric evaluation, ACEScg EXRs are transformed into Rec.709 scene-linear space to ensure photometric alignment with ground-truth ARRI footage.
Paired clips come from Poly Haven camera orbits [34]. Both sides start from sRGB Rec709 converted to ACEScg scene-linear frames, mapped through ACEScct T. The HDR canvas is clamped at 1.0. The SDR canvas is clamped at 0.55, then quantized, and passed through the camera-mimicking degradations of We use about 300 pairs. No ARRI footage is in the training set. The ARRI test set is 50 clips×49 frames at 1280X720; in-the-wild is 66 YouTube SDR clips with no HDR ground truth.
This section details the hyperparameter distributions, ACEScct masking formulas, and application probabilities for the SDR condition degradations described in Section 3.4.
Spatial blur degradation is applied with probability p=1. All degradations operate directly in ACEScct logarithmic space. For the SDR reference canvas, ACEScct values are clamped to y∈[0,0.55], where ywhite≈ 0.55 corresponds to diffuse white (x=1.0).
The highlight degradation Bhi(I) applies a Gaussian blur with standard deviation σ∼U(5,12) and kernel size k=2⌊ 2σ⌋+1. The shadow degradation Bsh(I) downsamples the log image by 2× using area interpolation followed by cubic upsampling. The degraded ACEScct image is blended back.
Contrast scaling is applied with probability p=0.30. The scaling factor f is uniformly sampled from U(1.0,1.5), anchored around the ACEScct midpoint 0.5:
Icontrast=clamp(0.5+(IACEScct-0.5)· f, 0, 0.55).(6)
All SDR reference sequences undergo H.264 video encoding. To simulate web video compression, 30% of reference clips are encoded with an elevated Constant Rate Factor CRF∼UniformDiscrete(23,35) (mean CRF≈ 29.3), while the remaining 70% use a high-quality baseline of CRF=12.
All methods are scored in Rec.709 scene-linear. ARRI ground truth is ARRI Wide Gamut converted to Rec.709. Our EXRs are ACEScg converted to Rec.709. DiffHDR [51] and Tedla et al. [39] already emit Rec.709 linear.
-3 because EV 0 is saturated on many clips. CLIP-IQA+ at EV +3 is a shadow check. There is no HDR ground truth on this split.Official 49-frame, 720× 1280 recipe on ARRI; 17-frame windows on in-the-wild. EXRs are Rec.709 linear. We used the published video recipe.
Run frame-wise on the same SDR frames, then stacked into a clip for video metrics. HDRTVNet++ targets HDR10 delivery, not scene-linear EXR.
On in-the-wild we run the official 17-frame, 704× 1280 recipe (empty prompt, 50 steps, seed 1). On ARRI we tile three windows at starts 0/16/32 (1-frame overlaps overwritten) to cover 49 frames; in-the-wild is a single 17-frame window. EXR’s are Rec.709 linear.
Figure 4 is an ActionVFX garage plate (no HDR ground truth) with the same methods as Tables 2–3. SDR, LumiVid, and DiffHDR are baked from scene-linear EXRs (ACEScg→Rec.709 for LumiVid; Rec.709-linear for DiffHDR) via sRGB(Y· 2EV). LEDiff is an image checkpoint (not a 49-frame video model).
Text-to-HDR uses the same LoRA with the SDR canvas dropped. The model denoises ACEScct tokens from text alone; we invert T to ACEScg EXR. Figure 10 compares LumiVid to X2HDR [44] in a text-toHDR mode, at EV -4, 0, and +4. X2HDR is an image model, so each frame is independent.
| LumiVid | ![]() | ![]() | ![]() |
| X2HDR | ![]() | ![]() | ![]() |
| LumiVid | ![]() | ![]() | ![]() |
| X2HDR | ![]() | ![]() | ![]() |
| LumiVid | ![]() | ![]() | ![]() |
| X2HDR | ![]() | ![]() | ![]() |
| EV − 4 -4 (highlights) | EV 0 0 (base) | EV + 4 +4 (shadows) |
-4 (highlights revealed), EV 0 (base), and EV +4 (shadows revealed). At EV +4, LumiVid preserves real spatial structure in dark and highlight regions, while X2HDR’s shadows and highlights collapse to a featureless plateau, brightening reveals less additional details. The EV -4 column shows a consistent advantage in highlight texture (sky, clouds, lit windows).@misc{korem2026hdr, title = {HDR Video Generation via Latent Alignment with Logarithmic Encoding}, author = {Korem, Naomi Ken and Oumoumad, Mohamed and Hagage, Omer and Gam, Amir and Ben Yosef, Matan and Cain, Harel and Jelercic, Urska and Bibi, Ofir and Inger, Yaron and Patashnik, Or and Cohen-Or, Daniel}, year = {2026}, eprint = {2604.11788}, doi = {10.48550/arXiv.2604.11788}, url = {https://ltx.io/research/hdr-video-generation} }
Explore other publications from the LTX Research team