Photographing long scenes with multi-viewpoint panoramas

Aseem Agarwala, Maneesh Agrawala, Michael Cohen, David Salesin, Richard Szeliski
2006 ACM SIGGRAPH 2006 Papers on - SIGGRAPH '06  
tographing long scenes with multi-viewpoint panoramas. In SIGGRAPH '06: ACM SIGGRAPH 2006 Papers, pages 853-861, New York, NY, USA, 2006. ACM The authors are presenting a new system to produce multi-viewpoint panoramas of long, roughly planar scenes from a set of photographs. This system is designed to exhibit four main properties: 1. Each object in the scene is rendered from a viewpoint roughly in front of it to avoid perspective distortion. 2. Composed of large regions of linear perspective
more » ... en from a viewpoint by a person. 3. Local perspective effects are evident. 4. The seams between these perspective regions do not draw attention. User begins by capturing a sequence of photographs that depicts the scene by walking along the scene and take handheld photographs (uses fisheye lens to insure a wide filed of view) every meter. In the first phase called preprocessing, this system remove radial distortion and recover the projection matrices of each camera to project the source images into a picture surface. The next step is to compensate for exposure variation between the source photographs, so they match better in overlapping regions. A brightness factor k i is assigned to each image I i and for two photographs we should have k i I i = k j I j for pixels that depicts the same geometry. The next step is picture surface selection, which has two steps itself: 1. Define a coordinate system of the recovered 3D scene. It can be done automatically by assuming the constant height of the camera and assigning the dimension of least variation to y-axis, of most variation to x-axis and the cross product to z-axis. It can also be done interactively by project the 3D scene points into the original source-photographs; then the user selects few projected points that lie along the desired axes. 2. Actually draw the curve in the xz plane that defines the picture surface; the system asks user to draw the picture surface as a polyline and then sweeps it up and down the y-axis to form a picture surface. Then the system samples the picture surface to form a regular 2D grid by projecting each sample of the surface into the source photograp and produces a simple average image, then user can perform wrapping and cropping. The panorama can be created now by choosing from among the possible viewpoints for each pixel. So far we created a series of n images I i of equivalent dimension, each represent the i's viewpoint. We can create a panorama by choosing a color for each pixel p = (p x , p y ) from one of the source images I i (p). The authors formulate an objective function that approximately measures the properties mentioned earlier, and minimize it using Markov Random Field (MRF) optimization. This objective function has three terms and a constraint. The first term reflects the first property, if pixel p chooses its color from I i , the heuristic is approximated as the 2D distance from p to p i . The second term encourages transition between different regions of linear perspective to be natural and seamless. The last term encourages the panorama to resemble the average image where the scene geometry intersects the picture surface. The constraint is that any one camera has a limited field of view and will not project to every pixel of the panorama, so we do not allow color selection from them. If the user is not satisfied with the result, the system allows for high-level interactive control. This system offers three types of strokes. "View selection" strokes allow a user to indicate that a certain viewpoint should be used in a certain area of the panorama. A "seam suppression" stroke indicates an object through which no seam should pass; we describe a novel technique for propagating this stroke across the different positions the object might take in the different viewpoints. Finally, an "inpainting" stroke allows the user to eliminate undesirable features such as power lines through inpainting. This paper tries to introduce a novel way of producing high-quality and high-resolution panoramas using casual capture methods. This is a major contribution as previous works required complex photo capturing techniques to accomplish this task. They introduced an objective function that describes some desirable properties of a multiviewpoint panorama. In addition, they proposed a novel technique for propagating user-drawn strokes that annotated 3D objects in the scene. This paper is generally well-written. I liked to see a comparison with another work (e.g., strip panoramas) by taking photos of the same scene and putting them side by side, this could help the reader to see the differences and feel them easier. Another weakness, as authors mentioned in the future works, is the depth discontinuity which is obvious in most of the provided results. Moreover, the automatic approach for producing panoramas seems to have some problems at the place of the seams in the results (some areas are stretched and some parts are removed).
doi:10.1145/1179352.1141966 fatcat:sqnzaolhrbfp5kkvzzhpgc2x3u