Back to Publications

HDR Video Generation via Latent Alignment with Logarithmic Encoding

License:
CC BY 4.0
Arxiv:
2604.11788
View on Arxiv

LumiVid generates HDR video from a single SDR input via ACEScct latent alignment and dual-canvas LoRA — no masks, text prompts, or reference images.

Naomi Ken Korem1,  Mohamed Oumoumad12,  Omer Hagage1,  Amir Gam1,  Matan Ben Yosef1,  Harel Cain1,  Urska Jelercic1,  Ofir Bibi1,  Yaron Inger1,  Or Patashnik3,  Daniel Cohen-Or3
1Lightricks  2Gear Productions  3Tel Aviv University
Back to Publications
License:
CC BY 4.0
Arxiv:
2604.11788

HDR Video Generation via Latent Alignment with Logarithmic Encoding

LumiVid generates HDR video from a single SDR input via ACEScct latent alignment and dual-canvas LoRA — no masks, text prompts, or reference images.

Naomi Ken Korem1,  Mohamed Oumoumad12,  Omer Hagage1,  Amir Gam1,  Matan Ben Yosef1,  Harel Cain1,  Urska Jelercic1,  Ofir Bibi1,  Yaron Inger1,  Or Patashnik3,  Daniel Cohen-Or3
1Lightricks  2Gear Productions  3Tel Aviv University
Abstract

High dynamic range (HDR) video captures continuous scene radiance essential for professional color grading and post-production workflows, yet generative video diffusion models remain bound to 8-bit standard dynamic range (SDR) formats. Existing adaptation methods either alter the VAE latent space or rely on auxiliary inputs like luminance masks, text prompts, or external reference images. However, relying on auxiliary inputs introduces manual overhead, makes the exact prompt hard to guess, and yields less stable results that often fail to remain consistent with the original footage.

We present LumiVid, a framework that generates HDR video directly from a single SDR input, without requiring masks, text prompts, or external reference images. Key to our method is mapping the HDR video into a logarithmic color space representation, aligning high-magnitude radiance with the existing latent space of a frozen video VAE without losing luminance details. Fine-tuning a dual-canvas LoRA adapter on just 300 synthetic paired clips enables data-efficient SDR-to-HDR translation while preserving temporal consistency. To prevent shortcut pixel copying during training and force the model to infer missing radiance and details from its learned priors, we introduce camera-mimicking condition degradations on the SDR reference input. Extensive evaluations show that LumiVid outperforms state-of-the-art baselines by +5.5 dB PU21-PSNR on cinema-grade ARRI footage, producing high-fidelity half-float EXR video with exceptional re-exposure latitude across extreme exposure pulls.

ImageVideo
SDR (Input)__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
HDR (Ours)__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
Figure 1: LumiVid: HDR video generation from SDR input. Given an input SDR image or video (top), our method produces HDR (bottom). Both rows use Mertens fusion of the same frame at multiple EVs. The image panel shows a mountain-road still at EV -2 and a sunset still at EV 0. The video panel shows two times from a rooftop clip at EV -1 and EV -3.

Introduction

High dynamic range (HDR) imagery captures scene radiance over a wide range of intensities, enabling faithful representation of highlights, shadows, and subtle illumination effects that are lost in standard imagery. This capability is essential for realistic rendering, post-production, professional color grading, and emerging display technologies.

Professional film and visual effects workflows rely on scene-referred data: a format where pixel values represent the actual physical light captured by the camera sensor. In this system, diffuse white (the light bouncing off a flat, non-shiny surface like a white sheet of paper) is set to a baseline value of 1.0. Brighter light isn’t capped at 1.0. Specular glare, car headlights, and direct sunlight extend unbounded into values of 10, 100, or even thousands. Holding onto real-world light values makes post-production reversible. Colorists adjust exposure in stops (the standard measurement of camera exposure, where each stop doubles or halves the light energy (Pixelnew=Pixel× 2stops). Since the video file retains data far above 1.0, dropping the exposure by a few stops reveals crisp details inside a glowing window instead of leaving a blown-out white blob. Shadow recovery also avoids color banding because dark tones were saved as continuous light levels. This linear math is also essential for realistically blending 3D CGI with live-action footage. Only at the end of the workflow is the footage converted into display-referred data, a set of image instructions tailored specifically to the physical limits of a monitor. For standard SDR screens, this meant squashing all light into a rigid 8-bit scale from 0 to 255, where 255 is the absolute brightest white the screen can produce. Modern HDR screens can display 10-bit or 12-bit color depths, but even they require a final display transform to map unbounded light values onto what the screen hardware can physically output.

This compression explains why video AI models struggle to generate native HDR. Video AI models are trained mostly on web videos stored in 8-bit SDR formats, where a piece of white paper and the sun are both squashed into the exact same ceiling of 255. These models learn exclusively from flattened, monitor-bound data, they lack an internal knowledge of how real physical light behaves beyond standard display limits.

A natural strategy for extending diffusion models to HDR is to leverage the visual structure, semantics, and temporal dynamics already learned by pretrained video models [19]. One common approach is to train a dedicated HDR encoder or fine-tune VAE decoders to emit linear radiance [41], creating an HDR-specific latent space. However, altering or replacing the encoder shifts the underlying latent manifold, forcing the downstream diffusion model to adapt to an unfamiliar space. This dependency demands massive HDR datasets and makes training complex and computationally expensive.

Drastically altering the latent space breaks compatibility with the model’s pretrained features. Keeping the HDR representation aligned with the model’s native latent space allows for data-efficient adaptation while preserving existing capabilities, such as video editing, inpainting, and controlled generation, directly in the HDR domain without needing to retrain those tools.

DiffHDR [51] has begun to explore aligning HDR with the model’s native latent space, but relies on external prompts and masks to synthesize missing highlights. In contrast, our method performs direct SDR-to-HDR translation conditioned only on the input video itself. We map scene-linear HDR into ACEScct, a standardized cinematic logarithmic color space (see Appendix B for color space fundamentals). Encoding this log video with the frozen SDR VAE yields latents already aligned with the pretrained manifold, allowing the generator to emit log-encoded HDR with lightweight fine-tuning and without a new encoder.

We train an in-context LoRA on a dual-canvas architecture, where SDR tokens condition the denoising of ACEScct HDR target tokens. 300 paired clips are sufficient: the LoRA learns a targeted dynamic-range conversion. To encourage the model to learn true HDR generation from an SDR source, we make the task even more challenging. Since SDR video already lacks detail in extreme luminance regions, we apply camera-mimicking degradations, such as aggressive clipping, heavy compression, and selective blur, to strip away even more information from the highlights and shadows. Combined with varied exposure shifts during training, this forces the network to rely on its internal priors to actively synthesize continuous, high-magnitude radiance rather than copying flat SDR pixels, eliminating the need for text prompts or luminance masks.

We evaluate on professional ARRI cinema footage and on 66 YouTube in-the-wild SDR videos. Both evaluation benchmarks comprise real cinematic footage featuring complex actor motion, dynamic lighting transitions, and non-rigid moving objects. Our findings show that HDR capabilities can be enabled in existing video models with high data efficiency, without architectural redesign or full retraining.

Our key contributions are:

  1. SDR-to-HDR generation: We generate HDR video directly from a pretrained video model using only the SDR video as an input, requiring no luminance masks, text prompts, or external reference images.
  2. Logarithmic color space alignment and data-efficient adaptation: Mapping HDR video into a logarithmic color space representation aligns high-magnitude radiance latents with the existing latent space of a frozen SDR VAE without losing intricate luminance details. Framing the process as a dynamic-range translation enables fine-tuning a dual-canvas LoRA adapter on just ∼300 synthetic paired clips while preserving temporal consistency. Despite training exclusively on synthetic camera orbits, our model generalizes seamlessly to real-world footage containing dynamic actor motion and complex scene transitions at inference.
  3. Camera-mimicking degradations and exposure augmentations: We combine contrast clipping, compression, highlight/shadow blur, and joint exposure shifts, alongside an asymmetric clamp ([0,0.55] on the SDR reference input vs. [0,1.0] on the HDR target), to prevent shortcut pixel copying during training, forcing the model to infer extreme high-magnitude highlights and deep shadow details from its learned priors. Video results, exposure pulls, and an overview are at https://hdr-lumivid.github.io/.

Related Work

Classical SDR-to-HDR Reconstruction

The challenge of capturing and manipulating high dynamic range content has long been a focal point of computational photography. Early foundations were built on multi-exposure HDR recovery [12] and photographic tone mapping for display [35]. A landmark in forward tone-mapping (HDR-to-SDR) was the work of Fattal et al. [15], who pioneered gradient-domain dynamic range compression. By selectively attenuating large luminance gradients (global contrast) while preserving small gradients (local detail), they demonstrated that integrating the modified field via a Poisson equation could compress dynamic range without introducing edge artifacts. This was further refined by Farbman et al. [14], who introduced edge-preserving multiscale decompositions based on Weighted Least Squares (WLS) optimization, providing a robust framework for detail manipulation that effectively eliminated the halo artifacts common in non-linear filtering.

While these classical operators focused on compressing scene-linear HDR for bounded displays, the complementary task of Inverse Tone Mapping (ITM) aims to expand SDR content back into unbounded radiance. ITM evolved from early heuristic expansion operators [5, 36] to modern deep CNN-based reconstruction architectures that reverse the camera pipeline or estimate linear radiance from single exposures [13, 32, 27]. A significant step toward modern generative approaches was taken by Marnerides et al. [31], whose work on Deep HDR Estimation moved beyond deterministic pixel mapping by utilizing generative detail hallucination for saturated regions. More recent architectures improve efficiency via frequency-domain processing [45] or leverage specialized sensing modalities such as event cameras [48].

A parallel line converts SDRTV to display-referred HDRTV (PQ, BT.2020) rather than scene-linear radiance. HDRTVNet [11] introduced a practical SDR-to-HDRTV pipeline and the HDRTV1K benchmark; HDRTVNet++ [10] extends it with adaptive global color mapping, local enhancement, and highlight refinement. HDRTVDM [18] adds degradation models and HDRTV4K. We use HDRTVNet++ as a per-frame baseline, but it targets 10-bit HDR display delivery rather than 16-bit scene-linear EXR files.

Extending to video, prior work focuses on temporal alignment and consistency using optical flow [23], coarse-to-fine alignment [9], or specialized sensing modalities [48]. Recent methods further address robustness to illumination changes [50] and real-time performance [46], supported by large-scale HDR video datasets [37].

These approaches treat SDR-to-HDR as a reconstruction problem, learning dedicated mappings under restricted imaging assumptions. In contrast, we treat HDR generation through the lens of distribution alignment: pretrained video diffusion models already capture rich priors over illumination, highlights, and shadows. The key challenge is to expose and unlock this knowledge by aligning HDR input/output distributions with the latent representations expected by the frozen model.

Generative & Diffusion-Based Image HDR Synthesis

Recent research has shifted toward leveraging generative models to synthesize missing radiance. Early works explored unsupervised HDR synthesis via GlowGAN, a StyleGAN variant [40] or applied conditional diffusion to multi-exposure image deghosting [47] and multi-exposure reconstruction [17]. Frame-level frameworks like BracketDiffusion [6] employ exposure-aware modeling to ensure consistency across simulated brackets, while other approaches adapt pretrained diffusion backbones through gain-map decomposition [16, 26] or zero-shot transfer [53]. LEDiff [41] enables text-to-HDR and LDR-to-HDR via exposure-aware latent fusion.

X2HDR [44] is concurrent image work. It adapts pretrained image diffusion models to HDR by operating in PU21 encoded perceptual space with a frozen VAE and LoRA fine-tuning—reflecting the view that HDR can be handled by mapping it into a compatible latent space. However, these methods rely primarily on display-oriented encodings and are restricted to single images. When applied frame-by-frame to video sequences, single-image diffusion models exhibit temporal flickering and lack cross-frame motion coherence.

SDR-to-HDR Video Diffusion Models

Video diffusion models have rapidly advanced from temporal extensions of image diffusion [21] to large-scale generative foundation models such as Stable Video Diffusion [8], CogVideoX [49], and Wan2.1 [43]. Modern architectures, including LTX-Video [20] and LTX-2 [19], combine highly compressed spatial-temporal video VAEs with scalable Diffusion Transformers (DiT) [33] and rotary position embeddings [38]. Low-Rank Adaptation (LoRA) [22] has become the standard mechanism for parameter-efficient adaptation of these backbones, as demonstrated by concurrent lighting estimation frameworks like LuxDiT [25].

Two recent works explore video diffusion backbones for SDR-to-HDR conversion. DiffHDR (Yu et al., 2026) [51] shares our use of a frozen VAE, log-space encoding, and LoRA adaptation on a video DiT. However, it formulates SDR-to-HDR as a masked radiance inpainting task where luminance masks mark clipped regions, and external text prompts or reference images specify what content to synthesize. The other one is Tedla et al. (2026) [39] Fully fine-tunes Wan2.2 to generate a linearized three-exposure SDR bracket (-4/0/+4 EV) via temporal latent concatenation, subsequently fusing the bracket with a learned per-frame merger network. All VAE operations remain in native SDR space, forming HDR only post-decoding via classical bracket merging.

Our framework, LumiVid, differs from both approaches. Unlike DiffHDR, LumiVid treats conversion as SDR to HDR process. The input SDR video is the only input signal, requiring no masks, text prompts, or reference images. Unlike Tedla et al., who keep generative priors in SDR and rely on multi-exposure bracket generation, we map scene-linear HDR directly into ACEScct log space. This allows a frozen VAE to represent high-magnitude radiance directly and enables a lightweight LoRA to emit 16-bit scene-linear HDR video in a single pass, without requiring an exposure ladder or secondary fusion network.

Method

__wf_reserved_inherit
Figure 2: LumiVid training overview: Scene-linear HDR is encoded with ACEScct and split. The HDR branch is passed through the frozen VAE; noise is added in latent space to form the diffusion target. The SDR condition is formed by the HDR-to-SDR luminance map and camera-mimicking degradations, then encoded by the same VAE. The two latents are concatenated as a dual canvas, and fed to the video DiT; only the LoRA adapters are trained, with a diffusion loss on the HDR latents.

Standard video diffusion models are bound to standard dynamic range (SDR) formats. We address SDR-to-HDR video generation by fine-tuning a Dual Canvas Lora adapter [7] on top of a frozen video diffusion model. The pretrained video that we are using is video DiT LTX-2.5 [19]. Given an SDR source video, the network generates a spatially and temporally aligned HDR target. During the training, the model gets clean SDR latents and noisy HDR target to denoise as a dual canvas . At inference, it receives the input SDR latents alongside Gaussian noise, iteratively denoising the noise into the final HDR video.

Our training recipe consists of three core components (Figure 2):

  1. ACEScct Log Working Space: Mapping unbounded scene-linear radiance into a cinematic log encoding aligned with the frozen VAE’s expected input range.
  2. Condition Degradations & Exposure Shifts: Applying camera-mimicking degradations and exposure shifts to the SDR canvas to erase shortcut pixels, forcing the model to infer extreme highlights and shadows.
  3. Dual-Canvas LoRA Adaptation: Fine-tuning a lightweight adapter on ∼ 300 paired clips to learn targeted dynamic-range translation.

ACEScct Log Working Space and Latent Alignment

To adapt a pretrained video diffusion model to HDR with lightweight fine-tuning, the model must operate on a latent distribution that closely matches what it was originally trained on. Pretrained video backbones contain learned priors about real-world physics, lighting, and motion acquired from web-scale SDR datasets. Since HDR video data is scarce, retraining the backbone or learning a new latent space would destroy these pretrained representations. Keeping the VAE encoder and decoder frozen ensures that we preserve the original model’s knowledge, enabling data-efficient adaptation.

However, directly encoding scene-linear HDR data with the frozen VAE fails. Unbounded, heavy-tailed RGB values deviate drastically from the VAE’s expected SDR input range, causing severe reconstruction artifacts. As shown in Table 1, measuring the direct VAE roundtrip (x→Encoder→Decoder→x) reconstruction error on linear RGB frames yields poor quality, resulting in a low PU21-PSNR of 13.99 dB and a high L2 error of 2.81. This low performance is expected since the VAE was trained on SDR data where the color range was 0-1, and not unbounded. A high roundtrip reconstruction error indicates that the VAE latents fail to faithfully capture the HDR video; because radiance details are lost during encoding, the decoder cannot accurately reconstruct the original scene. These degraded latents cannot serve as a viable target for diffusion training.

To bridge this representation gap while maintaining our frozen latent space, we map scene-linear HDR into ACEScct, a standardized cinematic logarithmic color space. ACEScct compresses unbounded radiance into a quasi-logarithmic scale aligned with the input statistics expected by the frozen VAE. Evaluating the VAE roundtrip reconstruction through the full ACEScct HDR target space (clamped at 1.0) yields an improvement, boosting the PU21-PSNR to 29.97 dB and reducing the L2 error to 0.69. By contrast, clamping the SDR reference condition at 0.55 (diffuse white) yields 23.02 dB PU21-PSNR, explicitly stripping highlights above white so the reference cannot leak high dynamic range codes into the condition canvas. We also show that clipping the ACEScct at 1.0 and then converting back to ACEScg results in almost no loss in the data with PU21-PSNR > 99.0

This confirms that the frozen SDR VAE already possesses sufficient capacity to encode and decode HDR content once the color distribution is properly aligned. By maintaining this alignment, we can teach the model to generate full HDR video while retaining the rich generative priors of the original model.

__wf_reserved_inherit
Figure 3: LumiVid Inference Overview. An SDR video is mapped with ACEScct and encoded by the frozen VAE to produce reference latents zref. These reference latents are concatenated as a dual canvas with noise and iteratively denoised by the Video DiT with LoRA adapters. The output latents are VAE-decoded and inverted through ACEScct to recover ACEScg scene-linear HDR video. The VAE and DiT remain frozen throughout; only the lightweight LoRA adapters were trained via flow matching.
InputPoly HavenARRI
PU21-PSNR ↑L2 ↓PU21-PSNR ↑L2 ↓
Linear unclipped13.992.8114.770.88
ACEScct 0.5523.022.8024.150.80
ACEScct 1.029.970.6930.170.16
ACEScct 1.0, no VAE>99.00∼ 0>99.00∼ 0

Logarithmic Formulation Let x denote scene-linear radiance in ACEScg. We map x into the normalized ACEScct logarithmic space via the piecewise transformation T(x):

T(x)=10.540· x+0.073x<2-7,log2(x)+9.7217.52x≥ 2-7.(1)

The linear region below x=2-7 avoids logarithmic encoding and stabilizes numerical representations in deep shadows. Following iterative denoising and VAE decoding, scene-linear radiance is reconstructed via the inverse transformation T-1(y) for an encoded log value y∈[0,1]:

T-1(y)=y-0.07310.540y<0.155,217.52· y-9.72y≥ 0.155.(2)

(Note: Constants above are rounded for readability; exact double-precision coefficients follow the standard ACEScct specification. [2])

Dual Canvas LoRA

We adapt a frozen LTX-2.5 video Diffusion Transformer (DiT) [19] using a rank-128 dual-canvas LoRA [7] trained with flow matching, keeping the VAE frozen. Following the dual-canvas formulation, the input sequence is constructed by concatenating two latent streams along the sequence dimension: a reference canvas containing clean (un-noised) SDR latents, and a target canvas containing noisy HDR latents. During denoising, the target HDR tokens directly attend to the clean SDR tokens through the DiT’s self-attention mechanism. To maintain precise spatial-temporal correspondence, the 3D Rotary Position Embeddings (RoPE) of the SDR reference tokens are aligned directly with those of the target HDR tokens. The flow-matching diffusion loss is computed only on the target HDR tokens.

This in-context conditioning allows the LoRA adapter to learn a localized dynamic-range mapping from the ACEScct SDR reference to the ACEScct HDR target. The frozen backbone already retains priors over video motion and physics, thus, the model does not need to learn video synthesis from scratch, enabling rapid convergence on just a few hundred paired clips (∼ 300). Text-to-HDR generation drops the SDR reference canvas and is detailed in Appendix J.

Synthetic Dataset and SDR Conditioning

High-quality HDR video data is scarce in the wild, thus, we construct a synthetic paired SDR-HDR video dataset by rendering synthetic camera orbits through high-resolution PolyHaven 360° HDRi environment maps [34]. We produced approximately 300 49-frame scene-linear HDR video clips. Each clip is converted to ACEScg and mapped into ACEScct log space via T(x).

While our training dataset uses camera orbits through static HDRi environments, all evaluation benchmarks (ARRI cinema footage and youtube in-the-wild clips) are conducted strictly on real-world video footage featuring dynamic real motion, including actor movements, non-rigid object dynamics, complex occlusions, and cinematic camera work, rather than synthetic camera orbits. Generalization from synthetic training data to real-world footage succeeds because motion and temporal priors remain in the frozen DiT backbone. The backbone handles scene dynamics and physics, while the LoRA adapter focuses exclusively on translating ACEScct SDR codes to ACEScct HDR codes. Training the LoRA on PolyHaven HDRI camera orbits provides rich photometric highlight and shadow variations while relying on the frozen backbone for temporal and motion dynamics.

Given the HDR rendered videos we can derive the corresponding SDR conditioning sequences used during training. In ACEScct log space, diffuse white (x=1.0) maps to T(1.0)≈ 0.55, while the upper boundary y=1.0 caps extreme scene radiance at T-1(1.0)≈ 223× diffuse white. To force the model to learn true dynamic-range translation from the SDR reference to the HDR target we use an asymmetric clamping strategy during training.

  • SDR reference: Projected to ACEScct and clamped to [0,0.55]. This strips highlight radiance above diffuse white from the reference input, ensuring the SDR condition carries no latent HDR codes.
  • HDR Target: Clamped to [0,1.0], exposing target latents to the full dynamic range up to ≈ 223× white.

This 0.55 vs. 1.0 split forces the network to infer extreme high-magnitude highlights (y∈(0.55,1.0]) from learned diffusion priors rather than leaking pixels from the conditioning canvas.

Camera-Mimicking Degradations and Exposure Augmentations

To encourage the model to synthesize true HDR and make the SDR-to-HDR training task more challenging, we apply three camera-mimicking degradation operators to the SDR reference canvas prior to VAE encoding:

  1. Luminance-Targeted Spatial Blur: Rather than applying uniform spatial blur, we selectively degrade extreme luminance regions in log space using softened selection masks. For clipped highlights near the SDR ceiling (y≈ 0.55), Gaussian blur simulates optical dispersion and lens bloom. To prevent the model from overfitting to a fixed blur pattern, the Gaussian standard deviation σ∈[5,12] and kernel size are dynamically randomized across clips. For crushed shadows near the log toe (y≈ 0.073), an area-downsampling and cubic-upsampling roundtrip strips fine spatial texture.
  2. Anchored Contrast Scaling: We scale contrast in ACEScct space via y=0.5+(y-0.5)· f. Anchoring around a fixed log midpoint (0.5) rather than a per-channel frame mean prevents white-balance and color-neutral drift while driving highlight and shadow values into clipping boundaries.
  3. H.264 Video Recompression: To model web-video compression artifacts, reference sequences undergo H.264 recompression with variable Constant Rate Factors (CRF∈[23,35]), introducing macroblocking and quantization noise directly into the SDR input.

Full hyperparameter distributions and implementation specifics are provided in Appendix E.

In addition to structural degradations, we apply joint exposure shifts across both the SDR condition and HDR target canvases. Simultaneously shifting both streams by shared random exposure offsets (spanning -2,0,2 EV stops) exposes the network to diverse luminance scales. This forces the model to recover both extreme highlights and deep shadow details across varied exposure levels while preserving spatial alignment between condition and target.

Together, these operations erase structural details in clamped or blurred regions, forcing the network to synthesize continuous physical radiance from its learned diffusion priors rather than reproducing input pixels. Consequently, the model generates smooth, unclipped specular gradients when pulling exposure down (EV-4/-3) and reveals coherent spatial texture when pushing shadows up (EV+3/+4).

Inference Pipeline

During inference, an input SDR video is converted from Rec.709 primaries to linear ACEScg, mapped via T(x) into the clamped ACEScct [0,0.55] and encoded by the frozen VAE. The Video Diffusion Transformer (DiT) with dual-canvas LoRA adapters performs iterative denoising in ACEScct latent space. Finally, decoded latents in ACEScct [0,1] are mapped back to scene-linear ACEScg via T-1(y) and saved as 16-bit half-float EXR files.

Experiments

Input (SDR)__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
HDRTVNet++__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
X2HDR__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
EV -7EV -3EV 0
DiffHDR__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
Tedla et al.__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
LumiVid__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
EV -7EV -3EV 0
Figure 4: HDR generation on an ActionVFX [3] garage plate, tone-mapped at EV -7, -3, and 0. Insets show the right headlight. At EV 0 the opening is blown out in every row; pulling exposure down shows what each method stored in the highlights. LumiVid recovers cloud structure through the door and a structured headlight. HDRTVNet++ barely expands the range. DiffHDR stays close to the SDR input (Table 3: Y99.9=0.84). Tedla et al. [39] (SDR2HDR) leaves little structure in the lamp at EV -7. X2HDR recovers some sky but clips the lamps.
Input (SDR)__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
DiffHDR__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
Tedla et al.__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
LumiVid__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
EV − 4 -4EV − 2 -2EV 0 0EV + 2 +2
Figure 5: HDR generation on the project-page sunset clip, tone-mapped at EV -4, -2, 0, and +2. EV 0 matches the SDR look; negative EVs show recovered highlights and EV +2 lifts the shadows. LumiVid keeps a compact solar disk and the cloud structure around it. DiffHDR stays close to the SDR input (Table 3: Y99.9=0.84). Tedla et al. leaves a small pale disk at EV -4.
EV -3EV 0EV +3EV -3EV 0EV +3
__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
EV -5EV -3EV 0EV -5EV -3EV 0
__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
EV -4EV 0EV +4EV -4EV 0EV +4
__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
SDR (Input)HDR
Figure 6: LumiVid qualitative results: For each scene, we compare the input SDR (left) with our generated HDR (right) across multiple exposure values (EV). Our method effectively recovers high-frequency details and synthesizes plausible radiance in clipped regions.

Experimental Setup

Datasets. We evaluate LumiVid on two benchmarks. ARRI Cinema Benchmark [4] which consist of 50 professional cinema clips with authentic 12-bit HDR ground truth. Input SDR videos are generated by forward tone-mapping the raw HDR plates to 8-bit Rec.709. These clips contain real-world cinematography, including motion, and dynamic lighting transitions. The second benchmark is In-the-Wild Benchmark: 66 diverse SDR video clips collected from YouTube. This benchmark tests generalization on real-world non-rigid motion (e.g., dynamic objects, human faces and motion, lighting transitions) without HDR ground truth.

We compare LumiVid against four state-of-the-art SDR-to-HDR baselines spanning both per-frame image models and native video diffusion backbones.

  1. HDRTVNet++ [10]: A per-frame inverse tone-mapping network designed for SDRTV-to-HDRTV translation. Frames are processed independently and stacked into video sequences. HDRTVNet++ targets display-referred HDR10 delivery rather than 16-bit scene-linear EXR radiance files.
  2. X2HDR [44]: A generative single-image SDR-to-HDR diffusion model operating in PU21 perceptual space. Applied frame-by-frame to video sequences.
  3. The closest native video baseline is DiffHDR [51], employing a frozen VAE, log encoding, and masked radiance inpainting on a video DiT, based on a textual prompt. However, relying on luminance masks and text prompts reformulates the task into generative inpainting rather than direct SDR-to-HDR translation, adding manual overhead and risking visual inconsistency.
  4. A concurrent SDR-to-HDR video diffusion method Tedla et al. [39] that fine-tunes Wan2.2 to predict linearized multi-exposure brackets for post merging.

All methods run at 720× 1280 resolution. We run 49-frame on ARRI and 17-frame on in-the-wild clips, outputting Rec.709 scene-linear EXR files. Tedla et al., works only on 17 frames, thus when running inference on ARRI benchmark, we run three 17-frame windows, that are tiled at starting frame indices 0, 16, and 32 (overwriting 1-frame overlaps) to cover 49 frames.

Evaluation Metrics. All generated EXR files are mapped to Rec.709 scene-linear space prior to metric calculation to ensure strict photometric alignment with the other baselines. We evaluate our method using common hdr video metrics, across four axes:

  1. Ground-Truth Referenced Fidelity: Can be measured only in the ARRI benchmark. Standard PSNR on unbounded linear radiance over-penalizes absolute errors in extreme highlights where human visual sensitivity is low, while ignoring perceptually critical errors in the darker areas. We therefore report, PU21-PSNR / SSIM [28] that computes PSNR and SSIM after mapping linear radiance into the perceptually uniform PU21 space, ensuring errors in shadow and mid-tone regions, where the eye is far more sensitive, are properly reflected in the fidelity score. We also report ColorVideoVDP [29] measures perceptual video quality and visible artifacts by modeling human visual sensitivity across spatial, temporal, and chromatic dimensions across varying dynamic ranges, Outputting a score on a 0–10 scale.
  2. Temporal Stability: To measure frame-to-frame coherence, we report F2F-PSNR, PSNR computed between consecutive PU21-encoded frames (yt and yt+1). We also report t-LPIPS [52] - LPIPS between consecutive PU21 frames. Higher F2F-PSNR and lower t-LPIPS indicate stable, flicker-free video generation.
  3. No-Reference Quality (In-the-Wild): On clips without ground truth, we assess output quality using MUSIQ [24] a multi-scale transformer image quality assessment (IQA) model and CLIP-IQA+ [42]. Since these models expect bounded 8-bit Rec.709 inputs, we evaluate them after applying explicit exposures. (EV-3 to inspect recovered highlight detail and EV+3 for shadow detail) followed by sRGB tone-mapping.
  4. Dynamic Range Expansion (Stops): To measure how far each method expands luminance above diffuse white, we compute scene-linear Rec.709 luminance Y (where Y=1.0 represents diffuse white). In a scene-referred frame only a very small part of the pixels should reach extreme high values, a light, the sun, a specular, a sparkle, while the rest of the frame sits between 0-1. We therefore read the tail with a high percentile rather than a statistic of the whole frame. We define the dynamic range in stops as: Stops=log2(Y99.9) (3) where Y99.9 is the 99.9th percentile of Y over all pixels in a clip. The percentile is high enough to fall on that small highlight set. We report the median over the 66 clips.

Flicker, highlight recovery, and video quality are best judged in the supplementary video and in a pairwise user study vs. HDRTVNet++, X2HDR and DiffHDR (§4.3).

Results

Table 2 reports ARRI GT-referenced fidelity and temporal stability. LumiVid leads in PU21-PSNR, ColorVideoVDP, and F2F. While HDRTVNet++ achieves equal or slightly better results on t-LPIPS, PU21-SSIM, its PU21-PSNR is much worse, indicating that its output fails to achieve true HDR dynamic range. Qualitative crops at EV-3 (highlights) and EV+4 (shadows) are shown in Figure 6. In-the-wild baselines are presented in Figures 4 and 5.

MethodPU21-PSNR ↑PU21-SSIM ↑ColorVideoVDP ↑F2F ↑t-LPIPS ↓
HDRTVNet++22.430.9357.2944.810.0057
X2HDR18.270.8183.1944.600.0143
DiffHDR24.280.7434.5740.360.0075
Tedla et al. [39]16.800.6874.7743.590.0070
LumiVid29.790.9157.9254.280.0059

Table 3 scores the in-the-wild benchmark. F2F / t-LPIPS for native video; MUSIQ and CLIP-IQA+ at EV -3 / +3 for highlights and shadows. LumiVid leads those columns.

LumiVid sits +2.8 stops above white at Y99.9, (6.7×) much higher than the other baselines. HDRTVNet++ is a mild display lift (1.7×). Flicker, highlight recovery, and the exposure pulls are on the project page. The comparisons are best viewed on an HDR display.

QualityRange
MethodF2F ↑t-LPIPS ↓MUSIQ-3 ↑CLIP-3 ↑CLIP+3 ↑Y99.9 ↑Stops ↑
HDRTVNet++37.720.06543.60.4120.3551.66+0.73
X2HDR38.190.06844.40.4260.3522.06+1.04
DiffHDR38.840.06540.80.4250.3850.84-0.26
Tedla et al. [39]36.550.06238.20.4030.3830.73-0.45
LumiVid45.150.05946.30.4340.4176.74+2.75

Perceptual User Study

Full-reference metrics like PU21-PSNR and ColorVideoVDP do not fully capture subjective visual quality or subtle temporal flickering. Thus, we conducted a pairwise 2AFC (Two-Alternative Forced Choice) perceptual user study comparing LumiVid against DiffHDR, X2HDR, and HDRTVNet++. A total of twenty one raters evaluated 30 video pairs each (10 scene pairs per baseline). The study evaluated three key dimensions: (1) temporal stability, color preservation, and flicker; (2) spatial detail and highlight texture recovery; and (3) overall visual quality. The EXR files were displayed as hlg files in three exposures, -3, 0, 3. To ensure good visual evaluation, all ratings were collected on a 1000-nit MacBook Pro display operating in native HDR mode. Figure 7 is flicker, detail, and overall quality. On this HDR display, raters prefer LumiVid to every baseline.

__wf_reserved_inherit
Figure 7: Pairwise preference on flicker, detail, and overall quality. Twenty one raters compared LumiVid to each baseline on ten clips drawn at random from the ARRI and in-the-wild test sets, on a 1000-nit MacBook display.

Ablation Studies

We conduct ablation experiments to isolate two design choices in our framework: (1) whether the generative diffusion prior of the Video DiT is necessary for high quality HDR recovery, or if a fine-tuned VAE decoder is sufficient; and (2) whether camera-mimicking condition degradations are required to prevent shortcut copying of 8-bit SDR pixel structures.

Isolating the Video DiT Backbone (VAE Decoder Ablation): A fundamental question is whether SDR-to-HDR video translation requires a generative diffusion process, or if it can be solved as a lightweight latent-to-pixel decoding task. Training a standalone encoder-decoder network from scratch would discard the rich representations learned by the pretrained VAE model and demand large-scale HDR video datasets that do not exist. Operating directly in the VAE’s latent space preserves these representations.

Furthermore, as established in §3.1, mapping scene-linear HDR into ACEScct allows the frozen VAE to represent log radiance without altering the latent manifold expected by the Video DiT (Table 1). This raises a compelling question: if the frozen SDR VAE encoder already produces latents that map well to ACEScct, can we simply fine-tune a VAE decoder to decode those latents directly into HDR frames in a single feed-forward pass?

To test this, we construct a VAE decoder-only baseline. We fine-tune the pixel decoder directly on ACEScct targets while keeping the VAE encoder frozen, completely omitting the Video DiT. At inference, SDR video is encoded and decoded to HDR in a single pass. This isolates whether the Video DiT is required for temporal consistency and high-magnitude radiance synthesis, or if log encoding plus a fine-tuned pixel decoder is sufficient.

Input (SDR)__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
VAE decoder__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
LumiVid__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
Figure 8: Decoder-only ablation vs. the main LoRA on four plates (Rec.709 preview of the HLG files, EV 0). The VAE decoder can lift range in the window, sky, and sun; but it does not put structure there. Faces, cloth, and foliage smooth, and those highlights clip. LumiVid is adding the HDR details and structure on top of the SDR input. Best to view these results in the attached project page.
SplitMetricVAE DecoderLumiVid (Main)
ARRI (n=50)PU21-PSNR ↑29.4329.79
ColorVideoVDP ↑7.607.92
F2F ↑46.3554.28
t-LPIPS ↓0.00820.0059
In-the-Wild (n=66)F2F ↑43.3345.15
t-LPIPS ↓0.0620.059
MUSIQ-343.246.3
CLIP-IQA+-30.4070.434
CLIP-IQA++30.3800.417

Fine-tuning only the VAE decoder for SDR-to-HDR translation fails to deliver high-fidelity, temporally stable HDR video. While a spatial VAE decoder can predictably re-map diffuse mid-tones, it lacks the generative capacity of the Video DiT backbone to synthesize unclipped radiance or enforce cross-frame temporal coherence. As detailed in Table 4, the limitations of this decoder-only baseline are evident across both ground-truth-referenced and in-the-wild evaluations.

Under a 3-stop highlight pull (EV-3), the VAE decoder fails to synthesize plausible structure in overexposed regions, producing flat, featureless clipping plateaus. This is directly reflected in in-the-wild no-reference metrics, where MUSIQ-3 drops from 46.3 (LumiVid) to 43.2 and CLIP-IQA+-3 falls from 0.434 to 0.407. Symmetrically, pushing shadow regions up by 3 stops (EV+3) causes CLIP-IQA++3 to drop from 0.417 to 0.380, demonstrating the decoder’s inability to reconstruct details in crushed darks. The VAE decoder exhibits significant temporal instability. On the ARRI benchmark, frame-to-frame fidelity (F2F) drops sharply from 54.28 to 46.35, while temporal flicker (t-LPIPS) worsens from 0.0059 to 0.0082. Qualitatively (Figure 8), whereas LumiVid leverages diffusion priors to selectively generate extreme peak radiance dictated by scene semantics, the VAE decoder merely expands pixel values uniformly—causing sky regions to oversmooth, specular highlights to smear into halos, and fine geometric edges to vanish. These results confirm that temporally consistent dynamic-range expansion requires the full generative capacity of the Video DiT backbone.

Impact of Camera-Mimicking Degradations: To force the model to infer unclipped radiance rather than copy input pixels, we apply contrast clipping, compression, and highlight/shadow blur to the SDR conditioning canvas during training. Table 5 evaluates the impact of these degradations. Training with no augmentation lowers PU21-PSNR by nearly 1.0 dB (28.83 vs. 29.79 dB) and degrades ColorVideoVDP (7.45 vs. 7.92). Without contrast clipping, the SDR reference retains quantized highlight steps; the network learns a shortcut by simply stretching 8-bit steps into linear space, resulting in severe posterization and temporal flickering. Combining contrast clipping, highlight/shadow blur, and joint exposure shifts, forces the Video DiT to rely on its diffusion prior to synthesize continuous, unclipped radiance.

ConfigPU21-PSNR ↑ColorVideoVDP ↑F2F ↑
Full augs (main)29.797.9254.28
No augmentation28.837.4549.14
Blur only (no contrast)29.117.3949.95

Conclusion

We presented LumiVid, an SDR-to-HDR method that generates temporally coherent wide range HDR video from a single SDR input via a video diffusion model backbone. We showed that the barrier is not model capacity but how we align HDR content with pretrained models. A video diffusion transformer trained only on SDR data already possesses rich knowledge about light transport and dynamic range. Unlocking this knowledge requires solving two problems: mapping HDR into the frozen VAE’s learned distribution, and forcing the model to learn to reconstruct extreme-luminance regions from its learned priors. With these in place, a LoRA adapter adding less than 1% parameters, trained for only 5,000 steps on ∼300 clips (∼8 hours, single GPU), produces temporally coherent float16 HDR video that outperforms the strongest dedicated SDR-to-HDR baseline. Several extensions are natural. Scaling the training data to include real-world HDR video captures would strengthen generalization. Incorporating perceptual HDR metrics such as HDR-VDP-3 [30] or ColorVideoVDP [29] as training objectives could further improve perceptual quality.

Our logarithmic alignment preserves the backbone’s native latent space, exciting direction for future work is extending this framework to other pretrained video capabilities, such as HDR video editing, HDR inpainting, and fine-grained visual control.

Acknowledgments

We would like to thank Alon Yaar and Alon Alpert from the LTX R&D group, and Omer Rabinovoch, Michel Angielczyk, and Yoav Lutzky from the LTX Creative group, for their guidance and support throughout the development of LumiVid. We are especially grateful to Jean Dominique Vandenberg (JD) for sharing his expertise in color science.

Appendix A Supplementary media

The project page, including the overview video, is the intended way to judge the results: https://hdr-lumivid.github.io/

Appendix B Color working space

Human color gamut, SDR, and HDR

The human color gamut is the boundary of human color vision, drawn as the CIE 1931 chromaticity diagram (Figure 9). The triangles inside it, spanned by red, green, and blue primaries, are the smaller subsets that a given screen, camera, or printer can actually reproduce.

Gamut is which chromaticities exist at all. Most SDR video is stored as Rec.709: a small triangle inside the full locus. Saturated LEDs, neon, and many camera-native colors lie outside it and are clipped onto the Rec.709 sides. An HDR working space uses a wider triangle so those chromaticities remain distinct instead of collapsing to the Rec.709 primaries.

Quantization is whether those values are a continuum or a palette. Consumer SDR is 8-bit integer per channel after a gamma curve: 256 discrete codes. Nearby radiances become the same code; an exposure pull or a grade shows banding. HDR is stored as scene-linear floating point (or 10–12-bit PQ on a display curve).

Light strength is how bright the sample is relative to a white surface in the scene. SDR is display-referred and clipped to a peak, historically about 100 nits. The sun, a headlamp, and a white wall can share the same code. HDR is scene-referred: we take x=1 as diffuse white and let highlights be many times that.

Most videos on the web are Rec.709 since Rec.709 was specified for CRT-era television: D65 white, those three primaries, and a peak the display could actually make. Cameras, broadcast, discs, and later phones and web codecs (H.264/H.265 SDR, YouTube) copied that contract because the device at the end of the chain was a Rec.709 display with an 8-bit pipeline. HDR panels exist now, but not at a scale close to Rec.709 video. The pretrained video models we adapt were trained on that Rec.709 corpus, so their VAE expects Rec.709-like statistics.

Rec.709 vs. ACEScg

Rec.709 is a display/camera encoding: D65, the small triangle in Figure 9. ACEScg is a scene-linear working space from the Academy Color Encoding System [1]: AP1 primaries -wider than Rec.709, inside AP0. (Rec.709 primaries lie inside AP1.), ACES white near D60, and linear RGB with x=1 equal to diffuse white and no upper bound. AP0 is the ACES2065-1 container gamut; its blue primary is imaginary (below the spectral locus), which is why the container can wrap Rec.709, AP1, and several camera encodings without clipping chromaticity. AP1 is the working gamut we actually compute in. ACEScg is not a display format: it is unbounded linear light on a wide triangle.

__wf_reserved_inherit
Figure 9: CIE xy chromaticity. Rec.709 is the small SDR triangle (D65): the gamut of almost all web and broadcast video. ACEScg is AP1 (dashed gold), a wider working triangle inside the AP0 container (solid gold). AP0’s blue primary is imaginary (below the spectral locus), so the container can wrap Rec.709 and AP1 without clipping chromaticity. The diagram shows chromaticity only; light strength (how far above white) is not plotted.

At inference we decode the SDR file with the sRGB EOTF and convert Rec.709 linear to ACEScg and then to ACEScct. The reverse path writes ACEScg EXR; metrics convert ACEScg to Rec.709 linear so every method is scored in the same space as the ARRI ground truth (Appendix F).

ACEScct

Linear ACEScg is a poor input to a frozen SDR VAE: the distribution is heavy-tailed and part of the pixels sits far above the [0,1] codes the VAE saw in training. ACEScct is a per-channel log of AP1.

Let x be an ACEScg linear RGB channel. The forward map is

T(x)=10.540237742x+0.072905534x≤ 0.0078125,log2(x)+9.7217.52x>0.0078125.(4)

The inverse is

T-1(y)=y-0.07290553410.540237742y≤ 0.15525114155,2y· 17.52-9.72y>0.15525114155.(5)

T(1)≈0.555 (diffuse white) and T-1(1)≈223. The linear toe below x=2-7 avoids log 0 in deep shadows.

Before the VAE we clamp ACEScct. On the SDR canvas the upper clamp is 0.55, just below T(1), so the condition cannot present highlights above white (T-1(0.55)≈0.94). On the HDR canvas the upper clamp is 1.0 (≈223× white).

The 0.55 vs. 1.0 split stops the LoRA from copying values that would otherwise sit in both ranges. Camera-mimicking degradations are applied on the SDR canvas after this clamp. ACEScct is a production log color space with a documented inverse.

Appendix C Training and Inference Implementation Details

Dual-Canvas LoRA Architecture

Following AVControl [7], we construct an in-context dual-canvas framework by concatenating the SDR reference canvas with the HDR target canvas in the spatial-temporal sequence dimension. The SDR condition canvas is encoded once by the frozen VAE and injected as reference tokens, while the HDR canvas contains the noisy target latents. Noisy latents are not concatenated along the channel dimension. For text-to-HDR generation, the SDR condition canvas is simply dropped (Appendix J). Training the LoRA in this dual-canvas fashion is exceptionally data-efficient because the backbone does not need to learn generative video synthesis from scratch; it merely learns a targeted dynamic-range translation from the ACEScct SDR sub-space to the full ACEScct HDR range.

Optimization Protocol

We train a rank-128 LoRA adapter on the frozen LTX-2.5 Video DiT backbone using flow matching for 5,000 steps on ∼300 paired 49-frame clips. The VAE encoder and decoder remain completely frozen throughout training. Optimization takes approximately eight hours on a single NVIDIA A100 GPU. The trainable LoRA parameters represent less than 1% of the total backbone parameters. At inference time, we use 8 denoising steps with standard flow matching sampling.

Inference Color Pipeline

During inference, the input SDR video file is decoded using the standard sRGB Electro-Optical Transfer Function (EOTF) to obtain scene-linear values, converted from Rec.709 color primaries to ACEScg scene-linear space, mapped into the log ACEScct SDR range [0,0.55] via T(x), and encoded into latent space by the frozen VAE. The Video DiT with rank-128 LoRA adapters performs iterative denoising directly in ACEScct latent space. Following VAE decoding, output latents in ACEScct [0,1.0] are inverted back to scene-linear ACEScg via T-1(y) and exported as 16-bit half-float EXR files. For metric evaluation, ACEScg EXRs are transformed into Rec.709 scene-linear space to ensure photometric alignment with ground-truth ARRI footage.

Appendix D Training data

Paired clips come from Poly Haven camera orbits [34]. Both sides start from sRGB Rec709 converted to ACEScg scene-linear frames, mapped through ACEScct T. The HDR canvas is clamped at 1.0. The SDR canvas is clamped at 0.55, then quantized, and passed through the camera-mimicking degradations of We use about 300 pairs. No ARRI footage is in the training set. The ARRI test set is 50 clips×49 frames at 1280X720; in-the-wild is 66 YouTube SDR clips with no HDR ground truth.

Appendix E Camera-Mimicking Degradation Implementation Details

This section details the hyperparameter distributions, ACEScct masking formulas, and application probabilities for the SDR condition degradations described in Section 3.4.

E.1 Luminance-Targeted Spatial Blur in ACEScct Space

Spatial blur degradation is applied with probability p=1. All degradations operate directly in ACEScct logarithmic space. For the SDR reference canvas, ACEScct values are clamped to y∈[0,0.55], where ywhite≈ 0.55 corresponds to diffuse white (x=1.0).

The highlight degradation Bhi(I) applies a Gaussian blur with standard deviation σ∼U(5,12) and kernel size k=2⌊ 2σ⌋+1. The shadow degradation Bsh(I) downsamples the log image by 2× using area interpolation followed by cubic upsampling. The degraded ACEScct image is blended back.

E.2 Anchored Contrast Scaling

Contrast scaling is applied with probability p=0.30. The scaling factor f is uniformly sampled from U(1.0,1.5), anchored around the ACEScct midpoint 0.5:

Icontrast=clamp(0.5+(IACEScct-0.5)· f, 0, 0.55).(6)

E.3 H.264 Recompression

All SDR reference sequences undergo H.264 video encoding. To simulate web video compression, 30% of reference clips are encoded with an elevated Constant Rate Factor CRF∼UniformDiscrete(23,35) (mean CRF≈ 29.3), while the remaining 70% use a high-quality baseline of CRF=12.

Appendix F Photometric protocol

All methods are scored in Rec.709 scene-linear. ARRI ground truth is ARRI Wide Gamut converted to Rec.709. Our EXRs are ACEScg converted to Rec.709. DiffHDR [51] and Tedla et al. [39] already emit Rec.709 linear.

Appendix G Evaluation metricks

  1. Ground-Truth Referenced Fidelity: PU21-PSNR/SSIM [28] PU21-PSNR/SSIM is PSNR (and SSIM) between ground-truth and generated frames after those frames are encoded with PU21. That encoding makes the PSNR better matched to the eye. In both frames the brightest values are very high; the eye is not very sensitive to differences in that range, and much more sensitive in the darks. Encoding to PU21 therefore gives a larger error to mistakes in the more sensitive parts of the frame. ColorVideoVDP. JOD [29] is a comparison to ground-truth, of how the clips would look on an HDR display rather than in encoded pixels. It models a 1000-nit panel, with 1.0 in the EXR shown as 100 nits (diffuse white), and scores short video windows (9 frames at 24 fps) on a 0–10 JOD scale. As with PU21, small errors in the brightest highlights count less than errors in darker regions; here that is built into the display model instead of into a PSNR.
  2. Temporal metrics. F2F is PSNR between consecutive PU21 frames. t-LPIPS is mean LPIPS between consecutive PU21 frames.
  3. No-reference (in-the-wild). MUSIQ and CLIP-IQA+ are reported at EV -3 because EV 0 is saturated on many clips. CLIP-IQA+ at EV +3 is a shadow check. There is no HDR ground truth on this split.

Appendix H Baseline recipes

DiffHDR

Official 49-frame, 720× 1280 recipe on ARRI; 17-frame windows on in-the-wild. EXRs are Rec.709 linear. We used the published video recipe.

HDRTVNet++ and X2HDR

Run frame-wise on the same SDR frames, then stacked into a clip for video metrics. HDRTVNet++ targets HDR10 delivery, not scene-linear EXR.

Tedla et al

On in-the-wild we run the official 17-frame, 704× 1280 recipe (empty prompt, 50 steps, seed 1). On ARRI we tile three windows at starts 0/16/32 (1-frame overlaps overwritten) to cover 49 frames; in-the-wild is a single 17-frame window. EXR’s are Rec.709 linear.

Appendix I In-the-wild baseline comparison

Figure 4 is an ActionVFX garage plate (no HDR ground truth) with the same methods as Tables 2–3. SDR, LumiVid, and DiffHDR are baked from scene-linear EXRs (ACEScg→Rec.709 for LumiVid; Rec.709-linear for DiffHDR) via sRGB(Y· 2EV). LEDiff is an image checkpoint (not a 49-frame video model).

Appendix J Text-to-HDR

Text-to-HDR uses the same LoRA with the SDR canvas dropped. The model denoises ACEScct tokens from text alone; we invert T to ACEScg EXR. Figure 10 compares LumiVid to X2HDR [44] in a text-toHDR mode, at EV -4, 0, and +4. X2HDR is an image model, so each frame is independent.

LumiVid__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
X2HDR__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
LumiVid__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
X2HDR__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
LumiVid__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
X2HDR__wf_reserved_inherit__wf_reserved_inherit__wf_reserved_inherit
EV − 4 -4 (highlights)EV 0 0 (base)EV + 4 +4 (shadows)
Figure 10: Detail revealed under exposure shifts text-to-HDR representative scenes. For each scene we show LumiVid (top row) and X2HDR (bottom row) at three exposures: EV -4 (highlights revealed), EV 0 (base), and EV +4 (shadows revealed). At EV +4, LumiVid preserves real spatial structure in dark and highlight regions, while X2HDR’s shadows and highlights collapse to a featureless plateau, brightening reveals less additional details. The EV -4 column shows a consistent advantage in highlight texture (sky, clouds, lit windows).
References
  1. Academy of Motion Picture Arts and Sciences (2014) ACES: academy color encoding system. Note: https://acescentral.com Cited by: Appendix B.
  2. Academy of Motion Picture Arts and Sciences (2016) ACEScct—a quasi-logarithmic encoding of ACES data for Use within Color Grading Systems. Technical report Technical Report Specification S-2016-001, Academy of Motion Picture Arts and Sciences, Beverly Hills, CA, USA. Cited by: §3.1.
  3. ActionVFX (2024) ActionVFX — pre-keyed vfx elements (commercial license). Note: https://www.actionvfx.com/ Cited by: Figure 4.
  4. ARRI (2024) ARRI sample footage archive. Note: https://www.arri.com/en/learn-help/learn-help-camera-system/camera-sample-footage-list Cited by: §4.1.
  5. F. Banterle, P. Ledda, K. Debattista, A. Chalmers, and M. Bloj (2007) A framework for inverse tone mapping. The Visual Computer 23 (7), pp. 467–478. Cited by: §2.1.
  6. M. Bemana, K. Myszkowski, H. Seidel, and T. Ritschel (2025) Bracket diffusion: HDR image generation by consistent LDR denoising. Computer Graphics Forum. Cited by: §2.2.
  7. M. Ben-Yosef, T. Halperin, N. K. Korem, M. Salama, H. Cain, A. Joseph, A. Chen, U. Jelercic, and O. Bibi (2026) AVControl: efficient framework for training audio-visual controls. arXiv preprint arXiv: 2603.24793. Cited by: Appendix C, §3.2, §3.
  8. A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts, et al. (2023) Stable video diffusion: scaling latent video diffusion models to large datasets. arXiv preprint arXiv: 2311.15127. Cited by: §2.3.
  9. G. Chen, C. Chen, S. Guo, Z. Liang, K. K. Wong, and L. Zhang (2021) HDR video reconstruction: a coarse-to-fine network and a real-world benchmark dataset. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2502–2511. Cited by: §2.1.
  10. X. Chen, Z. Li, Z. Zhang, J. Ren, Y. Liu, J. He, Y. Qiao, J. Zhou, and C. Dong (2023) Towards efficient sdrtv-to-hdrtv by learning from image formation. arxiv 2023. arXiv preprint arXiv: 2309.04084. Cited by: §2.1, item 1.
  11. X. Chen, Z. Zhang, J. S. Ren, L. Tian, Y. Qiao, and C. Dong (2021) A new journey from sdrtv to hdrtv. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4500–4509. Cited by: §2.1.
  12. P. E. Debevec and J. Malik (1997) Recovering high dynamic range radiance maps from photographs. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp. 369–378. Cited by: §2.1.
  13. G. Eilertsen, J. Kronander, G. Denes, R. K. Mantiuk, and J. Unger (2017) HDR image reconstruction from a single exposure using deep CNNs. In ACM Transactions on Graphics (Proc. SIGGRAPH Asia), Vol. 36. Cited by: §2.1.
  14. Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. In ACM Transactions on Graphics (Proc. SIGGRAPH), Cited by: §2.1.
  15. R. Fattal, D. Lischinski, and M. Werman (2002) Gradient domain high dynamic range compression. In ACM Transactions on Graphics (Proc. SIGGRAPH), Cited by: §2.1.
  16. Y. Guan, R. Xu, Y. Liao, M. Yao, L. Wang, and Z. Xiong (2025) HDR image generation via gain-map-decomposed diffusion. In ICCV, Cited by: §2.2.
  17. Y. Guan, R. Xu, M. Yao, R. Gao, L. Wang, and Z. Xiong (2024) Diffusion-promoted hdr video reconstruction. In European Conference on Computer Vision, pp. 20–38. Cited by: §2.2.
  18. C. Guo, L. Fan, Z. Xue, and X. Jiang (2023) Learning a practical sdr-to-hdrtv up-conversion using new dataset and degradation models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 22231–22241. Cited by: §2.1.
  19. Y. HaCohen, B. Brazowski, N. Chiprut, Y. Bitterman, A. Kvochko, A. Berkowitz, D. Shalem, D. Lifschitz, D. Moshe, E. Porat, et al. (2026) LTX-2: efficient joint audio-visual foundation model. arXiv preprint arXiv: 2601.03233. Cited by: §1, §2.3, §3.2, §3.
  20. Y. HaCohen, N. Chiprut, B. Brazowski, D. Shalem, D. Moshe, E. Richardson, E. Levin, G. Shiran, N. Zabari, O. Gordon, P. Panet, S. Weissbuch, V. Kulikov, Y. Bitterman, Z. Melumian, and O. Bibi (2025) LTX-Video: realtime video latent diffusion. arXiv preprint arXiv: 2501.00103. Cited by: §2.3.
  21. J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet (2022) Video diffusion models. In NeurIPS, Cited by: §2.3.
  22. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In ICLR, Cited by: §2.3.
  23. N. K. Kalantari and R. Ramamoorthi (2019) Deep hdr video from sequences with alternating exposures. In Computer graphics forum, Vol. 38, pp. 193–205. Cited by: §2.1.
  24. J. Ke, Q. Wang, Y. Wang, P. Milanfar, and F. Yang (2021) Musiq: multi-scale image quality transformer. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5128–5137. Cited by: item 3.
  25. R. Liang, K. He, Z. Gojcic, I. Gilitschenski, S. Fidler, N. Vijaykumar, and Z. Wang (2025) LuxDiT: lighting estimation with video diffusion transformer. In SIGGRAPH Asia, Cited by: §2.3.
  26. Y. Liao, Y. Guan, R. Xu, J. Li, S. Sun, and Z. Xiong (2025) Learning gain map for inverse tone mapping. In The Thirteenth International Conference on Learning Representations, Cited by: §2.2.
  27. Y. Liu, W. Lai, Y. Chen, Y. Kao, M. Yang, Y. Chuang, and J. Huang (2020) Single-image hdr reconstruction by learning to reverse the camera pipeline. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1651–1660. Cited by: §2.1.
  28. R. K. Mantiuk and M. Ashraf (2021) PU21: a novel perceptually uniform encoding for adapting existing quality metrics for HDR. In Picture Coding Symposium (PCS), Cited by: item 1, item 1.
  29. R. K. Mantiuk A. Chubarau et al. (2024) ColorVideoVDP: a visual difference predictor for image, video and display distortions. In ACM SIGGRAPH, Cited by: item 1, item 1, §5.
  30. R. K. Mantiuk, R. Weerakkody, S. Andriani, and M. Mrak (2023) HDR-VDP-3: a multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and normal content. arXiv preprint arXiv: 2304.13625. Cited by: §5.
  31. D. Marnerides, T. Bashford-Rogers, and K. Debattista (2021) Deep HDR hallucination for inverse tone mapping. Sensors 21 (12), pp. 4032. Cited by: §2.1.
  32. D. Marnerides, T. Bashford-Rogers, J. Hatchett, and K. Debattista (2018) Expandnet: a deep convolutional neural network for high dynamic range expansion from low dynamic range content. In Computer graphics forum, Vol. 37, pp. 37–49. Cited by: §2.1.
  33. W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In ICCV, Cited by: §2.3.
  34. Poly Haven (2024) Poly Haven — free high-dynamic-range environment maps. Note: https://polyhaven.com/hdris Cited by: Appendix D, §3.3.
  35. E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda (2002) Photographic tone reproduction for digital images. In ACM SIGGRAPH, pp. 267–276. Cited by: §2.1.
  36. A. G. Rempel, M. Trentacoste, H. Seetzen, H. D. Young, W. Heidrich, L. Whitehead, and G. Ward (2007) Ldr2hdr: on-the-fly reverse tone mapping of legacy video and photographs. ACM transactions on graphics (TOG) 26 (3), pp. 39–es. Cited by: §2.1.
  37. Y. Shu, L. Shen, X. Hu, M. Li, and Z. Zhou (2024) Towards real-world hdr video reconstruction: a large-scale benchmark dataset and a two-stage alignment network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2879–2888. Cited by: §2.1.
  38. J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu (2024) RoFormer: enhanced transformer with rotary position embedding. Neurocomputing 568, pp. 127063. Cited by: §2.3.
  39. S. Tedla, F. Banterle, T. Canham, K. Raja, D. B. Lindell, K. N. Kutulakos, J. Li, F. Li, and D. Iso (2026) Generating hdr video from sdr video. arXiv preprint arXiv: 2605.14703. Cited by: Appendix F, §2.3, Figure 4, item 4, Table 2, Table 3.
  40. C. Wang, A. Serrano, X. Pan, B. Chen, K. Myszkowski, H. Seidel, C. Theobalt, and T. Leimkühler (2023) Glowgan: unsupervised learning of hdr images from ldr images in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10509–10519. Cited by: §2.2.
  41. C. Wang, Z. Xia, T. Leimkühler, K. Myszkowski, and X. Zhang (2025) LEDiff: latent exposure diffusion for HDR generation. In CVPR, Cited by: §1, §2.2.
  42. J. Wang, K. C. Chan, and C. C. Loy (2023) Exploring clip for assessing the look and feel of images. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37, pp. 2555–2563. Cited by: item 3.
  43. WanTeam et al. (2025) Wan: open and advanced large-scale video generative models. arXiv preprint arXiv: 2503.20314. Cited by: §2.3.
  44. R. Wu, W. Su, K. Ma, J. Liao, and R. K. Mantiuk (2026) X2HDR: HDR image generation in a perceptually uniform space. arXiv preprint arXiv: 2602.04814. Cited by: Appendix J, §2.2, item 2.
  45. G. Xu, Q. Hou, and M. Cheng (2024) Dual frequency transformer for efficient sdr-to-hdr translation. Machine Intelligence Research 21 (3), pp. 538–548. Cited by: §2.1.
  46. G. Xu, Y. Wang, J. Gu, T. Xue, and X. Yang (2024) HDRFlow: real-time HDR video reconstruction with large motions. In CVPR, Cited by: §2.1.
  47. Q. Yan, T. Hu, Y. Sun, H. Tang, Y. Zhu, W. Dong, L. Van Gool, and Y. Zhang (2023) Toward high-quality hdr deghosting with conditional diffusion models. IEEE Transactions on Circuits and Systems for Video Technology 34 (5), pp. 4011–4026. Cited by: §2.2.
  48. Y. Yang, J. Han, J. Liang, I. Sato, and B. Shi (2023) Learning event guided high dynamic range video reconstruction. In CVPR, Cited by: §2.1, §2.1.
  49. Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, et al. (2024) CogVideoX: text-to-video diffusion models with an expert transformer. arXiv preprint arXiv: 2408.06072. Cited by: §2.3.
  50. Y. Ye, N. Zhang, Y. Zhao, H. Cao, and R. Wang (2024) Deep video inverse tone mapping based on temporal clues. In CVPR, Cited by: §2.1.
  51. Z. Yu, L. Ma, M. He, L. Isikdogan, Y. Xu, D. Smirnov, P. Salamanca, D. Mi, P. Delgado, N. Yu, et al. (2026) DiffHDR: re-exposing ldr videos with video diffusion models. arXiv preprint arXiv: 2604.06161. Cited by: Appendix F, §1, §2.3, item 3.
  52. R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang (2018) The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, Cited by: item 2.
  53. R. Zhu, S. Xu, P. Liu, S. Li, Y. Lu, D. Niu, Z. Liu, Z. Meng, Z. Li, X. Chen, and Y. Fan (2024) Zero-shot structure-preserving diffusion model for high dynamic range tone mapping. In CVPR, Cited by: §2.2.
BibTeX

@misc{korem2026hdr, title = {HDR Video Generation via Latent Alignment with Logarithmic Encoding}, author = {Korem, Naomi Ken and Oumoumad, Mohamed and Hagage, Omer and Gam, Amir and Ben Yosef, Matan and Cain, Harel and Jelercic, Urska and Bibi, Ofir and Inger, Yaron and Patashnik, Or and Cohen-Or, Daniel}, year = {2026}, eprint = {2604.11788}, doi = {10.48550/arXiv.2604.11788}, url = {https://ltx.io/research/hdr-video-generation} }

Table of Contents:

Discover more

Explore other publications from the LTX Research team

APEX: Asynchronous Prefix Denoising for Extreme Compression

Pairs a hierarchical latent space with asynchronous denoising over channel groups, enabling extreme compression that outperforms prior methods on ImageNet-512 under 64x and 128x compression.
Read Paper
Source:
arXiv
Source:
This is some text inside of a div block.
Oct 9th, 2026

HDR Video Generation via Latent Alignment with Logarithmic Encoding

LumiVid generates HDR video from a single SDR input via ACEScct latent alignment and dual-canvas LoRA — no masks, text prompts, or reference images.
Read Paper
Source:
arXiv
Source:
This is some text inside of a div block.
Oct 7th, 2026

Reference-Driven Multi-Speaker Audio Scene Generation from In-the-Wild Priors

Conditions a text-to-audio flow-matching model on multiple reference voices and a free-form prompt to generate natural multi-speaker audio scenes with real-world ambient texture.
Read Paper
Source:
arXiv
Source:
This is some text inside of a div block.
Jun 17th, 2026

JUST-DUB-IT : Video Dubbing via Joint Audio-Visual Diffusion

A single-model approach that adapts a joint audio-visual diffusion foundation model for video dubbing via a lightweight LoRA, replacing complex task-specific pipelines.
Read Paper
Source:
arXiv
Source:
This is some text inside of a div block.
May 11th, 2026

AVControl: Efficient Framework for Training Audio-Visual Controls

A lightweight, extendable framework built on LTX-2 that trains each video and audio control modality — depth, pose, camera, audio — as a separate LoRA, with no architectural changes.
Read Paper
Source:
arXiv
Source:
This is some text inside of a div block.
Mar 25th, 2026

LTX-2 : Efficient Joint Audio-Visual Foundation Model

An open-source asymmetric dual-stream transformer (14B video + 5B audio) that generates high-quality, temporally synchronized audiovisual content in a single unified pass.
Read Paper
Source:
arXiv
Source:
This is some text inside of a div block.
Jan 6th, 2026

LoRA Training for Text-to-Video Models: A Practical Guide to Fine-Tuning SOTA Video Generation

A practical SIGGRAPH Asia course guiding developers through fine-tuning state-of-the-art text-to-video models with LoRA.
Read Paper
Source:
ACM
Source:
This is some text inside of a div block.
Dec 14th, 2025

LTX-Video : Realtime Video Latent Diffusion

A transformer-based latent diffusion model that tightly integrates the Video-VAE and denoiser, reaching a 1:192 compression ratio to generate video faster than real time — with open weights.
Read Paper
Source:
arXiv
Source:
This is some text inside of a div block.
Dec 30th, 2024