Filters








497 Hits in 1.3 sec

Optical Flow in Mostly Rigid Scenes [article]

Jonas Wulff, Laura Sevilla-Lara, Michael J. Black
2017 arXiv   pre-print
The optical flow of natural scenes is a combination of the motion of the observer and the independent motion of objects. Existing algorithms typically focus on either recovering motion and structure under the assumption of a purely static world or optical flow for general unconstrained scenes. We combine these approaches in an optical flow algorithm that estimates an explicit segmentation of moving objects from appearance and physical constraints. In static regions we take advantage of strong
more » ... nstraints to jointly estimate the camera motion and the 3D structure of the scene over multiple frames. This allows us to also regularize the structure instead of the motion. Our formulation uses a Plane+Parallax framework, which works even under small baselines, and reduces the motion estimation to a one-dimensional search problem, resulting in more accurate estimation. In moving regions the flow is treated as unconstrained, and computed with an existing optical flow method. The resulting Mostly-Rigid Flow (MR-Flow) method achieves state-of-the-art results on both the MPI-Sintel and KITTI-2015 benchmarks.
arXiv:1705.01352v1 fatcat:qp7nfsubejd4ndbxb62hbftigm

Modeling Blurred Video with Layers [chapter]

Jonas Wulff, Michael Julian Black
2014 Lecture Notes in Computer Science  
Fig. 1 : When computing optical flow from motion blurred video (a), existing methods [44] fail at object boundaries ((b) and (e), red). Our method is able to accurately estimate optical flow (c), deblurred frames (d), and object boundaries ((e), green). Abstract. Videos contain complex spatially-varying motion blur due to the combination of object motion, camera motion, and depth variation with finite shutter speeds. Existing methods to estimate optical flow, deblur the images, and segment the
more » ... cene fail in such cases. In particular, boundaries between differently moving objects cause problems, because here the blurred images are a combination of the blurred appearances of multiple surfaces. We address this with a novel layered model of scenes in motion. From a motion-blurred video sequence, we jointly estimate the layer segmentation and each layer's appearance and motion. Since the blur is a function of the layer motion and segmentation, it is completely determined by our generative model. Given a video, we formulate the optimization problem as minimizing the pixel error between the blurred frames and images synthesized from the model, and solve it using gradient descent. We demonstrate our approach on synthetic and real sequences.
doi:10.1007/978-3-319-10599-4_16 fatcat:rre7ovcpafguzmpo2mgnqc2zdq

Seeing What a GAN Cannot Generate [article]

David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba
2019 arXiv   pre-print
Despite the success of Generative Adversarial Networks (GANs), mode collapse remains a serious issue during GAN training. To date, little work has focused on understanding and quantifying which modes have been dropped by a model. In this work, we visualize mode collapse at both the distribution level and the instance level. First, we deploy a semantic segmentation network to compare the distribution of segmented objects in the generated images with the target distribution in the training set.
more » ... fferences in statistics reveal object classes that are omitted by a GAN. Second, given the identified omitted object classes, we visualize the GAN's omissions directly. In particular, we compare specific differences between individual photos and their approximate inversions by a GAN. To this end, we relax the problem of inversion and solve the tractable problem of inverting a GAN layer instead of the entire generator. Finally, we use this framework to analyze several recent GANs trained on multiple datasets and identify their typical failure cases.
arXiv:1910.11626v1 fatcat:zuesge2t4jcznb3gacpih6qbfi

Temporal Interpolation as an Unsupervised Pretraining Task for Optical Flow Estimation [article]

Jonas Wulff, Michael J. Black
2018 arXiv   pre-print
The difficulty of annotating training data is a major obstacle to using CNNs for low-level tasks in video. Synthetic data often does not generalize to real videos, while unsupervised methods require heuristic losses. Proxy tasks can overcome these issues, and start by training a network for a task for which annotation is easier or which can be trained unsupervised. The trained network is then fine-tuned for the original task using small amounts of ground truth data. Here, we investigate frame
more » ... terpolation as a proxy task for optical flow. Using real movies, we train a CNN unsupervised for temporal interpolation. Such a network implicitly estimates motion, but cannot handle untextured regions. By fine-tuning on small amounts of ground truth flow, the network can learn to fill in homogeneous regions and compute full optical flow fields. Using this unsupervised pre-training, our network outperforms similar architectures that were trained supervised using synthetic optical flow.
arXiv:1809.08317v1 fatcat:guug6lvdabbcfiz7x3m37463yu

Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space [article]

Jonas Wulff, Antonio Torralba
2020 arXiv   pre-print
Modern Generative Adversarial Networks are capable of creating artificial, photorealistic images from latent vectors living in a low-dimensional learned latent space. It has been shown that a wide range of images can be projected into this space, including images outside of the domain that the generator was trained on. However, while in this case the generator reproduces the pixels and textures of the images, the reconstructed latent vectors are unstable and small perturbations result in
more » ... cant image distortions. In this work, we propose to explicitly model the data distribution in latent space. We show that, under a simple nonlinear operation, the data distribution can be modeled as Gaussian and therefore expressed using sufficient statistics. This yields a simple Gaussian prior, which we use to regularize the projection of images into the latent space. The resulting projections lie in smoother and better behaved regions of the latent space, as shown using interpolation performance for both real and generated images. Furthermore, the Gaussian model of the distribution in latent space allows us to investigate the origins of artifacts in the generator output, and provides a method for reducing these artifacts while maintaining diversity of the generated images.
arXiv:2009.06529v1 fatcat:d6uqys2ugzfp3b4kuphgljd5ca

Polyp Segmentation in NBI Colonoscopy [chapter]

Sebastian Gross, Manuel Kennel, Thomas Stehle, Jonas Wulff, Jens Tischendorf, Christian Trautwein, Til Aach
2009 Bildverarbeitung für die Medizin 2009  
Endoscopic screening of the colon (colonoscopy) is performed to prevent cancer and to support therapy. During intervention colon polyps are located, inspected and, if need be, removed by the investigator. We propose a segmentation algorithm as a part of an automatic polyp classification system for colonoscopic Narrow-Band images. Our approach includes multi-scale filtering for noise reduction, suppression of small blood vessels, and enhancement of major edges. Results of the subsequent edge
more » ... ction are compared to a set of elliptic templates and evaluated. We validated our algorithm on our polyp database with images acquired during routine colonoscopic examinations. The presented results show the reliable segmentation performance of our method and its robustness to image variations.
doi:10.1007/978-3-540-93860-6_51 dblp:conf/bildmed/GrossKSWTTA09 fatcat:uwr5clqwprguxeryldnqmqkbvm

Optical Flow in Mostly Rigid Scenes

Jonas Wulff, Laura Sevilla-Lara, Michael J. Black
2017 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
Figure 1 : Overview. From three frames (a) our method computes a segmentation of the scene into static (red) and moving (blue) regions (b), the depth structure of the scene (c) , and the optical flow (d). (e) shows ground truth flow. Abstract The optical flow of natural scenes is a combination of the motion of the observer and the independent motion of objects. Existing algorithms typically focus on either recovering motion and structure under the assumption of a purely static world or optical
more » ... low for general unconstrained scenes. We combine these approaches in an optical flow algorithm that estimates an explicit segmentation of moving objects from appearance and physical constraints. In static regions we take advantage of strong constraints to jointly estimate the camera motion and the 3D structure of the scene over multiple frames. This allows us to also regularize the structure instead of the motion. Our formulation uses a Plane+Parallax framework, which works even under small baselines, and reduces the motion estimation to a one-dimensional search problem, resulting in more accurate estimation. In moving regions the flow is treated as unconstrained, and computed with an existing optical flow method. The resulting Mostly-Rigid Flow (MR-Flow) method achieves state-of-the-art results on both the MPI-Sintel and KITTI-2015 benchmarks.
doi:10.1109/cvpr.2017.731 dblp:conf/cvpr/WulffSB17 fatcat:ldlx422wmzbwlnr54so3ltwgtq

Estimating the number of clusters via a corrected clustering instability

Jonas M. B. Haslbeck, Dirk U. Wulff
2020 Computational statistics (Zeitschrift)  
We improve instability-based methods for the selection of the number of clusters k in cluster analysis by developing a corrected clustering distance that corrects for the unwanted influence of the distribution of cluster sizes on cluster instability. We show that our corrected instability measure outperforms current instability-based measures across the whole sequence of possible k, overcoming limitations of current insability-based methods for large k. We also compare, for the first time,
more » ... -based and model-free approaches to determining cluster-instability and find their performance to be comparable. We make our method available in the R-package cstab.
doi:10.1007/s00180-020-00981-5 pmid:33088024 pmcid:PMC7550318 fatcat:co6a5kome5bsdes5jyxinpleru

A Naturalistic Open Source Movie for Optical Flow Evaluation [chapter]

Daniel J. Butler, Jonas Wulff, Garrett B. Stanley, Michael J. Black
2012 Lecture Notes in Computer Science  
Ground truth optical flow is difficult to measure in real scenes with natural motion. As a result, optical flow data sets are restricted in terms of size, complexity, and diversity, making optical flow algorithms difficult to train and test on realistic data. We introduce a new optical flow data set derived from the open source 3D animated short film Sintel. This data set has important features not present in the popular Middlebury flow evaluation: long sequences, large motions, specular
more » ... ions, motion blur, defocus blur, and atmospheric effects. Because the graphics data that generated the movie is open source, we are able to render scenes under conditions of varying complexity to evaluate where existing flow algorithms fail. We evaluate several recent optical flow algorithms and find that current highly-ranked methods on the Middlebury evaluation have difficulty with this more complex data set suggesting further research on optical flow estimation is needed. To validate the use of synthetic data, we compare the image-and flow-statistics of Sintel to those of real films and videos and show that they are similar. The data set, metrics, and evaluation website are publicly available.
doi:10.1007/978-3-642-33783-3_44 fatcat:clxg5ymr3beo5bhwbjdnv4qpvy

Dynamic Distortion Correction for Endoscopy Systems with Exchangeable Optics [chapter]

Thomas Stehle, Michael Hennes, Sebastian Gross, Alexander Behrens, Jonas Wulff, Til Aach
2009 Bildverarbeitung für die Medizin 2009  
Endoscopic images are strongly affected by lens distortion caused by the use of wide angle lenses. In case of endoscopy systems with exchangeable optics, e.g. in bladder endoscopy or sinus endoscopy, the camera sensor and the optics do not form a rigid system but they can be shifted and rotated with respect to each other during an examination. This flexibility has a major impact on the location of the distortion centre as it is moved along with the optics. In this paper, we describe an
more » ... for the dynamic correction of lens distortion in cystoscopy which is based on a one time calibration. For the compensation, we combine a conventional static method for distortion correction with an algorithm to detect the position and the orientation of the elliptic field of view. This enables us to estimate the position of the distortion centre according to the relative movement of camera and optics. Therewith, a distortion correction for arbitrary rotation angles and shifts becomes possible without performing static calibrations for every possible combination of shifts and angles beforehand.
doi:10.1007/978-3-540-93860-6_29 dblp:conf/bildmed/StehleHGBWA09 fatcat:w4duzorwlfh2bbmmniefttzldy

Lessons and Insights from Creating a Synthetic Optical Flow Benchmark [chapter]

Jonas Wulff, Daniel J. Butler, Garrett B. Stanley, Michael J. Black
2012 Lecture Notes in Computer Science  
With the MPI-Sintel Flow dataset, we introduce a naturalistic dataset for optical flow evaluation derived from the open source CGI movie Sintel. In contrast to the well-known Middlebury dataset, the MPI-Sintel Flow dataset contains longer and more varied sequences with image degradations such as motion blur, defocus blur, and atmospheric effects. Animators use a variety of techniques that produce pleasing images but make the raw animation data inappropriate for computer vision applications if
more » ... ed "out of the box". Several changes to the rendering software and animation files were necessary in order to produce data for flow evaluation and similar changes are likely for future efforts to construct a scientific dataset from an animated film. Here we distill our experience with Sintel into a set of best practices for using computer animation to generate scientific data for vision research.
doi:10.1007/978-3-642-33868-7_17 fatcat:tn3vhlmhmbcyvefy6yjjnwywzq

Using latent space regression to analyze and leverage compositionality in GANs [article]

Lucy Chai, Jonas Wulff, Phillip Isola
2021 arXiv   pre-print
., 2019; Wulff & Torralba, 2020 ).  ... 
arXiv:2103.10426v2 fatcat:obeyyn22lndc3f5v7xvffprbke

Correspondence estimation from non-rigid motion information

Jonas Wulff, Thomas Lotz, Thomas Stehle, Til Aach, J. Geoffrey Chase, Benoit M. Dawant, David R. Haynor
2011 Medical Imaging 2011: Image Processing  
The DIET (Digital Image Elasto Tomography) system is a novel approach to screen for breast cancer using only optical imaging information of the surface of a vibrating breast. 3D tracking of skin surface motion without the requirement of external markers is desirable. A novel approach to establish point correspondences using pure skin images is presented here. Instead of the intensity, motion is used as the primary feature, which can be extracted using optical flow algorithms. Taking sequences
more » ... multiple frames into account, this motion information alone is accurate and unambiguous enough to allow for a 3D reconstruction of the breast surface. Two approaches, direct and probabilistic, for this correspondence estimation are presented here, suitable for different levels of calibration information accuracy. Reconstructions show that the results obtained using these methods are comparable in accuracy to marker-based methods while considerably increasing resolution. The presented method has high potential in optical tissue deformation and motion sensing.
doi:10.1117/12.877896 dblp:conf/miip/WulffLSAC11 fatcat:lyyqcytzdrbyreymlqee5tw6ya

Learning to See by Looking at Noise [article]

Manel Baradad, Jonas Wulff, Tongzhou Wang, Phillip Isola, Antonio Torralba
2022 arXiv   pre-print
Acknowledgments: Manel Baradad was supported by the LaCaixa Fellowship and Jonas Wulff was supported by a grant from Intel Corp.  ... 
arXiv:2106.05963v3 fatcat:6hx5lqvk6ffyxl2a7nc5o6g4ze

Olfactory and chemosomatosensory function in pregnant women assessed with event-related potentials

Jonas K. Olofsson, Daniel A. Broman, Marianne Wulff, Mikael Martinkauppi, Steven Nordin
2005 Physiology and Behavior  
The purpose of the study was to better understand past findings of nasal chemosensory hypersensitivity in pregnant women by recording chemosensory event-related potentials (CSERPs) for information about cortical neuronal allocation (amplitudes) and temporal processing (latencies) of three concentrations of pyridine ranging from predominantly olfactory to trigeminal in activation in 15 pregnant and 15 nonpregnant women. CSERP components of primarily sensory (N1 and P2) and cognitive (P3) origin
more » ... ere evaluated. The results displayed no group differences in either N1 or P2 amplitudes or latencies, but tendencies of larger amplitudes and shorter latencies for P3 in pregnant women. This implies that pregnant women's hypersensitivity may more likely be referred to cognitive than sensory processing. D
doi:10.1016/j.physbeh.2005.07.022 pmid:16112693 fatcat:nczdcijkpzb6lhr2cdbzkxwgk4
« Previous Showing results 1 — 15 out of 497 results