Filters








5 Hits in 1.4 sec

Omnimatte: Associating Objects and Their Effects in Video [article]

Erika Lu, Forrester Cole, Tali Dekel, Andrew Zisserman, William T. Freeman, Michael Rubinstein
2021 arXiv   pre-print
In this work, we take a step towards solving this novel problem of automatically associating objects with their effects in video.  ...  Our model is trained only on the input video in a self-supervised manner, without any manual labels, and is generic -- it produces omnimattes automatically for arbitrary objects and a variety of effects  ...  This work was supported in part by an Oxford-Google DeepMind Graduate Scholarship and a Royal Society Research Professorship. We thank Weidi Xie for assisting with object removal baselines.  ... 
arXiv:2105.06993v2 fatcat:nf2tn3426zhh5gaqxonna5juaa

Omnimatte: Associating Objects and Their Effects in Video

Erika Lu, Forrester Cole, Tali Dekel, Andrew Zisserman, William T. Freeman, Michael Rubinstein
2021 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
We pose a novel problem: automatically associating subjects in videos with 'effects' related to them in the scene.  ...  The associated elements can be other objects attached to the subject or moving with it, or complex effects such as shadows, reflections, smoke, or ripples the subject creates in water.  ...  This work was supported in part by an Oxford-Google DeepMind Graduate Scholarship and a Royal Society Research Professorship. We thank Weidi Xie for assisting with object removal baselines.  ... 
doi:10.1109/cvpr46437.2021.00448 fatcat:hecplgtmcja37iws5u4pktoxpy

Text2LIVE: Text-Driven Layered Image and Video Editing [article]

Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kasten, Tali Dekel
2022 arXiv   pre-print
Given an input image or video and a target text prompt, our goal is to edit the appearance of existing objects (e.g., object's texture) or augment the scene with visual effects (e.g., smoke, fire) in a  ...  We present a method for zero-shot, text-driven appearance manipulation in natural images and videos.  ...  [26, 27] estimate omnimattes -RGBA layers that contain a target subject along with their associated scene effects.  ... 
arXiv:2204.02491v2 fatcat:nfm5h3tvdbe3rolskl5crzmuju

The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editing [article]

Dawit Mureja Argaw, Fabian Caba Heilbron, Joon-Young Lee, Markus Woodson, In So Kweon
2022 arXiv   pre-print
Our benchmark suite focuses on video editing tasks, beyond visual effects, such as automatic footage organization and assisted video assembling.  ...  This work introduces the Anatomy of Video Editing, a dataset, and benchmark, to foster research in AI-assisted video editing.  ...  Lu, E., Cole, F., Dekel, T., Zisserman, A., Freeman, W.T., Rubinstein, M.: Omnimatte: associating objects and their effects in video.  ... 
arXiv:2207.09812v2 fatcat:atwu3uk34rf2tezic2ze2m2x7q

Towards Real-Time Video Inpainting

Aron Ingruber, Walter Kropatsch, Jiri Hladuvka
2022
However, it can also be used to get rid of superimposed text or to remove certain objects or subjects from a video.  ...  This work proposes a modular real-time video inpainting method which is based on a regular video inpainting method and evaluates it with respect to different inpainting scenarios and compares it to the  ...  Instead it outputs omnimattes which are essentially alpha channel images that separate an objects and its effect from the video.  ... 
doi:10.34726/hss.2022.88909 fatcat:kpeovdo7trgcxgvbjt4x2yspkm