Laplacian Patch-Based Image Synthesis
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Patch-based image synthesis has been enriched with global optimization on the image pyramid. Successively, the gradient-based synthesis has improved structural coherence and details. However, the gradient operator is directional and inconsistent and requires computing multiple operators. It also introduces a significantly heavy computational burden to solve the Poisson equation that often accompanies artifacts in non-integrable gradient fields. In this paper, we propose a patch-based synthesis
... sing a Laplacian pyramid to improve searching correspondence with enhanced awareness of edge structures. Contrary to the gradient operators, the Laplacian pyramid has the advantage of being isotropic in detecting changes to provide more consistent performance in decomposing the base structure and the detailed localization. Furthermore, it does not require heavy computation as it employs approximation by the differences of Gaussians. We examine the potentials of the Laplacian pyramid for enhanced edge-aware correspondence search. We demonstrate the effectiveness of the Laplacian-based approach over the state-of-the-art patchbased image synthesis methods. * Corresponding author ture from the boundary to the interior such that the image gradients, the first derivatives of image intensity, measure directional changes of intensity around edges. The image pyramid, which denotes an image representation based on multiscale signals, has been widely used as a typical practice for enhancing structural coherence when completing missing regions in patch-based synthesis  . Recently, the image gradients in each level of the image pyramid are used for enhancing edge structure in addition to coherent patchbased synthesis  . Even though the latest approach of combining gradients and the image pyramid has improved the structural coherence and details in inpainting, the gradient operator is directional and thus requires twofold greater computation of multiple operators, and it introduces a significantly heavy computational burden to solve the Poisson equation. Furthermore, it presents inconsistency often with artifacts in non-integrable gradient fields. The Laplacian operator, which is the divergence of gradients of image intensity, takes advantage of being isotropic and invariant to rotation (Figure 1c ). In addition, coordinates of the Laplacian correspond to those of the edges, being well aligned to represent the image structure over edges. The Laplacian pyramid allows us to decompose the base and detail structure of an image into different spatial frequency components that can preserve structure upon decomposition. This representation has been used in many applications such as image blend/fusion, enhancement, and denoising. However, to the best of our knowledge, the potentials of the Laplacian pyramid has not been intensively exploited in previous coherent patch-based image synthesis. The proposed method is the first work that combines the Laplacian with patch-based synthesis of global coherence. In this paper, we examine the properties of the Laplacian pyramid for image completion and describe our edge-aware patch-based synthesis using a Laplacian pyramid. Related Work Proposed first by Bertalmio et al. , inpainting refers to the task of filling in or completing holes, or missing or corrupted regions in images. Inpainting is classified into two categories: diffusion-based and exemplar-based methods.