Abstract

Image demoiréing poses one of the most formidable challenges in image restoration, primarily due to the unpredictable and anisotropic nature of moiré patterns. Limited by the quantity and diversity of training data, current methods tend to overfit to a single moiré domain, resulting in performance degradation for new domains and restricting their robustness in real-world applications. In this paper, we propose a universal image demoiréing solution, UniDemoiré, which has superior generalization capability. Notably, we propose innovative and effective data generation and synthesis methods that can automatically provide vast high-quality moiré images to train a universal demoiréing model. Our extensive experiments demonstrate the cutting-edge performance and broad potential of our approach for generalized image demoiréing.

UniDemoiré Framework

The generalization ability of SOTA demoiréing models is greatly limited by the scarcity of data. Therefore, we mainly face two challenges to obtain a universal model with improved generalization capability: To obtain a vast amount of 1) diverse and 2) realistic-looking moiré data. Notice that traditional moiré image datasets contain real data, but continuously expanding their size to involve more diversity is extremely time-consuming and impractical. While current synthesized datasets/methods struggle to synthesize realistic-looking moiré images.


Hence, to tackle these challenges, we introduce a universal solution, UniDemoiré. The data diversity challenge is solved by collecting a more diverse moiré pattern dataset and presenting a moiré pattern generator to increase further pattern variations. Meanwhile, the data realistic-looking challenge is undertaken by a moiré image synthesis module. Finally, our solution can produce realistic-looking moiré images of sufficient diversity, substantially enhancing the zero-shot and cross-domain performance of demoiréing models.


Moiré Pattern Dataset

We propose to collect a moiré pattern dataset rather than a moiré image dataset, with no need for image alignment and can easily synthesize multiple moiré counterparts of a single natural image.

A detailed comparison of ours and others is shown in the Table below:

Comparisons of different moiré datasets. The "R" denotes the real dataset, and the "S" denotes the synthetic dataset.

Capturing Process

We capture videos of real-world moiré patterns on a pure white screen with a mobile phone to minimize color distortion in the moiré patterns. After recording, frames are uniformly extracted from each video to constitute our dataset.

Data collection setup (left), and examples of moiré patterns in our dataset captured at different zoom rates and screen panel (middle), and our generated patterns (right).

A video example with the settings above.


Data Diversity

To enhance pattern diversity, we build our dataset by considering additional factors that influence moiré formation, which were overlooked in previous moiré datasets, including Zooming Rate, Camera Types, CMOS, and Screen Panel Types. Besides, we doubled the number of mobile devices and display screens compared to existing datasets.

The mobile phone we apply to get the moiré patterns.

The screen we apply to get the moiré patterns.

Moiré Pattern Generation

We propose to use diffusion models to further sample more diverse moiré patterns by sufficiently learning the structural, textural, and color representations of real moiré patterns.

Data Preprocessing

Data preprocessing for moiré pattern generation.

Learning Moiré Patterns in the Latent Space

We notice that plenty of pixels in the moiré pattern appear pure white. This leads to a polarization in the pixel distribution of the moiré pattern images, where informative data is concentrated in a few pixels with high values while the rest contains little information.

Based on this observation, we choose to compress the moiré pattern into the latent space through an autoencoder for a more compact and efficient representation of its structural, textural, and color information. For better stability and controllability, we utilize the Latent Diffusion Model to effectively model the complex distribution of the moiré pattern in the latent space.

Visualization of sampled patches using our Moiré Pattern Generator.

Moiré Image Synthesis

Overview of the Moiré Image Synthesis stage (a). It involves a Moiré Image Blending module (b) for initial moiré image synthesis and a Tone Refinement Network (c) to refine for more realistic results.

Via data collection and generation, we obtain a vast number of diverse moiré patterns. Then, we need to composite moiré patterns with clean images to form moiré images. To make the synthesized images realistic-looking, We first create handcraft rules to produce initial moiré images in the Moiré Image Blending (MIB) module, then design a Tone Refinement Network (TRN) to further faithfully replicate the color and brightness variations observed in real scenes that cannot be fully formulated in those handcraft rules. Finally, the entire network is trained using a weighted compound of perception loss, color differentiable RGB-uv histogram loss and total variation regularizer.

Visualization of our intermediate synthetic results. The final synthesis of TRN best resembles the real moiré images in contrast and brightness distortions.

Image Demoiréing

Our contributions mainly lie in the above three stages. Then, diverse and realistic-looking data synthesized by our solution can be seamlessly integrated with demoiréing models to improve their performance.


Experimental Results


Zero-Shot Demoiréing with Synthesized Data Only

We first demonstrate demoiréing results on real moiré images trained on purely synthesized data by SOTA moiré synthesis methods.

To avoid data overlap in training sets and test sets, we have collected a comprehensive Mixed High-Resolution Natural Image Dataset (MHRNID), based on which, moiré images are synthesized for training demoiréing models.

Examples of the MHRNID dataset.

Quantitative comparisons can be found in Table below:

Quantitative results of zero-shot demoiréing trained with synthesized data only. "†" indicates UnDem uses moiré patterns retrieved from real data in TIP for inference. "‡" indicates UnDem uses our generated moiré pattern for inference.

From the quantitative perspective, our method substantially outperforms all other approaches. We attribute our superiority to the diversity and realism of our synthetic data. Such high-quality data by our UniDemoiré enables the demoiréing model to learn moiré characteristics better, improving performance in removing unseen moiré artifacts.

Cross-Dataset Evaluation

We then demonstrate our ability to improve the performance of demoiréing models across domains. Quantitative results are shown in Table below:

Quantitative results of cross-dataset evaluations.

As shown, thanks to the realistic and diverse synthesized data, our method outperforms all previous methods across every experiment.

More Qualitative Comparisons



Acknowledgements

This work is supported by NSFC (No.62206173), Shanghai Frontiers Science Center of Human-centered Artificial Intelligence (ShangHAI), MoE Key Laboratory of Intelligent Perception and Human-Machine Collaboration (KLIP-HuMaCo). This work is also partially supported by HKU-SCF FinTech Academy, HKRGC Theme-based research scheme project T35-710/20-R, and SZ-HK-Macau Technology Research Programme #SGDX20210823103537030.

BibTeX

@misc{yang2025unidemoire,
  author    = {Zemin Yang, Yujing Sun, Xidong Peng, Siu Ming Yiu, Yuexin Ma},
  title     = {UniDemoiré: Towards Universal Image Demoiréing with Data Generation and Synthesis},
  year      = {2025},
  eprint    = {2502.06324},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url={https://arxiv.org/abs/2502.06324},
}