Filters








1,798 Hits in 7.9 sec

Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward [article]

Kaiyang Zhou, Yu Qiao, Tao Xiang
2018 arXiv   pre-print
To train our DSN, we propose an end-to-end, reinforcement learning-based framework, where we design a novel reward function that jointly accounts for diversity and representativeness of generated summaries  ...  During training, the reward function judges how diverse and representative the generated summaries are, while DSN strives for earning higher rewards by learning to produce more diverse and more representative  ...  Acknowledgments We thank Ke Zhang and Wei-Lun Chao for discussions of details of their paper (Zhang et al. 2016b) .  ... 
arXiv:1801.00054v3 fatcat:hl2dexakjfa4zbptukr3buzfve

Deep Reinforcement Learning for Unsupervised Video Summarization With Diversity-Representativeness Reward

Kaiyang Zhou, Yu Qiao, Tao Xiang
2018 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
To train our DSN, we propose an end-to-end, reinforcement learning-based framework, where we design a novel reward function that jointly accounts for diversity and representativeness of generated summaries  ...  During training, the reward function judges how diverse and representative the generated summaries are, while DSN strives for earning higher rewards by learning to produce more diverse and more representative  ...  Acknowledgments We thank Ke Zhang and Wei-Lun Chao for discussions of details of their paper (Zhang et al. 2016b)  ... 
doi:10.1609/aaai.v32i1.12255 fatcat:nziaj4wwunbsbfdu4cpyn3ytyi

Ultrasound Video Summarization using Deep Reinforcement Learning [article]

Tianrui Liu, Qingjie Meng, Athanasios Vlontzos, Jeremy Tan, Daniel Rueckert, Bernhard Kainz
2020 arXiv   pre-print
Our approach is framed as reinforcement learning problem and produces agents focusing on the preservation of important diagnostic information.  ...  Video is an essential imaging modality for diagnostics, e.g. in ultrasound imaging, for endoscopy, or movement assessment.  ...  In this paper, we present an ultrasound imaging summarization method using deep reinforcement learning. We show effectiveness for the example of fetal ultrasound screening.  ... 
arXiv:2005.09531v1 fatcat:7smho4n6lber3ccxzwvvfac6yi

Weakly Supervised Video Summarization by Hierarchical Reinforcement Learning [article]

Yiyan Chen, Li Tao, Xueting Wang, Toshihiko Yamasaki
2020 arXiv   pre-print
Conventional video summarization approaches based on reinforcement learning have the problem that the reward can only be received after the whole summary is generated.  ...  With the guide of the subgoal, the worker predicts the importance scores for video frames in the subtask by policy gradient according to both global reward and innovative defined sub-rewards to overcome  ...  ACKNOWLEDGMENTS This work was partially financially supported by the Grants-in-Aid for Scientific Research Numbers JP19H05590 and JP19K20289.  ... 
arXiv:2001.05864v2 fatcat:mdkkrc75zjex3jnvk7blnqemqe

AudViSum: Self-Supervised Deep Reinforcement Learning for Diverse Audio-Visual Summary Generation

Sanjoy Chowdhury, Aditya Patra, Subhrajyoti Dasgupta, Ujjwal Bhattacharya
2021 British Machine Vision Conference  
To this end, we introduce a novel self-supervised audio-visual summarization network AudViSum, that leverages both audio and visual information and employs Deep Reinforcement Learning to reward the model  ...  To ensure diverse summary generation we report the top-3 summaries for each video.  ...  [58] proposed an unsupervised diversity-representativeness reward to guide the RL agent for video summarization. Taking cue from them we introduce a weighted reward function.  ... 
dblp:conf/bmvc/ChowdhuryPDB21 fatcat:qbzd2my6hreipl3xjzt55fkdoq

Interp-SUM: Unsupervised Video Summarization with Piecewise Linear Interpolation

Ui-Nyoung Yoon, Myung-Duk Hong, Geun-Sik Jo
2021 Sensors  
To train the video summarization network, we exploit a reinforcement learning-based framework with an explicit reward function.  ...  This paper addresses the problem of unsupervised video summarization. Video summarization helps people browse large-scale videos easily with a summary from the selected frames of the video.  ...  Recently, the reinforcement learning (RL) based unsupervised video summarization method outperformed in results with an explicit reward function to select keyframes [4] .  ... 
doi:10.3390/s21134562 fatcat:jjn655pcybbj5ove65lphehzae

Spatial attention model‐modulated bi‐directional long short‐term memory for unsupervised video summarisation

Rui Zhong, Diyang Xiao, Shi Dong, Min Hu
2021 Electronics Letters  
summarisation, deep reinforcement-deep summarization network (DR-DSN), which comprises a novel feedback reward with the aspects of diversity and representativeness.  ...  Given the excellent performance of reinforcement learning in video representation, Zhou [6] subsequently presented an end-to-end long short-term memory (LSTM)-based unsupervised learning method for video  ...  summarisation, deep reinforcement-deep summarization network (DR-DSN), which comprises a novel feedback reward with the aspects of diversity and representativeness.  ... 
doi:10.1049/ell2.12111 fatcat:ovrvf6dskbh65dra5cxgnfbqwm

Video Summarization Using Deep Neural Networks: A Survey [article]

Evlampios Apostolidis, Eleni Adamantidou, Alexandros I. Metsai, Vasileios Mezaris, Ioannis Patras
2021 arXiv   pre-print
After presenting the motivation behind the development of technologies for video summarization, we formulate the video summarization task and discuss the main characteristics of a typical deep-learning-based  ...  This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization.  ...  using a diversity-representativeness reward.  ... 
arXiv:2101.06072v2 fatcat:7mozntfhdrf3lkw6pwcr5v2rpu

Deep Reinforcement Learning for Query-Conditioned Video Summarization

Yujia Zhang, Michael Kampffmeyer, Xiaoguang Zhao, Min Tan
2019 Applied Sciences  
After that, a deep reinforcement learning-based summarization network (SummNet) is developed to provide personalized summaries by integrating relatedness, representativeness and diversity rewards.  ...  Query-conditioned video summarization requires to (1) find a diverse set of video shots/frames that are representative for the whole video, and that (2) the selected shots/frames are related to a given  ...  For example, Zhou and Qiao [25] develop a deep reinforcement learning-based summarization network with a diversity-representativeness reward to generate summaries, and achieve a good performance on generic  ... 
doi:10.3390/app9040750 fatcat:5nsbvbpxtjgbfejvm2xgjsxgba

Unsupervised multi-latent space reinforcement learning framework for video summarization in ultrasound imaging [article]

Roshan P Mathews, Mahesh Raveendranatha Panicker, Abhilash R Hareendranathan, Yale Tung Chen, Jacob L Jaremko, Brian Buchanan, Kiran Vishnu Narayan, Kesavadas C, Greeta Mathews
2021 arXiv   pre-print
We propose a new unsupervised reinforcement learning (RL) framework with novel rewards that facilitates unsupervised learning avoiding tedious and impractical manual labelling for summarizing ultrasound  ...  Our new paradigm for video summarization is capable of delivering classification labels and segmentation of key landmarks for each of the summarized keyframes.  ...  The authors are grateful to Compute Canada and the NVIDIA and CDAC for giving access to the PARAM SIDDHI AI system for providing the computing resources for the project.  ... 
arXiv:2109.01309v1 fatcat:urzvzwunczegppchpvy73vp4dq

AC-SUM-GAN: Connecting Actor-Critic and Generative Adversarial Networks for Unsupervised Video Summarization

Evlampios Apostolidis, Eleni Adamantidou, Alexandros I. Metsai, Vasileios Mezaris, Ioannis Patras
2020 IEEE transactions on circuits and systems for video technology (Print)  
This paper presents a new method for unsupervised video summarization.  ...  Moreover, the introduced criterion for choosing the best model after the training ends, enables the automatic selection of proper values for parameters of the training process that are not learned from  ...  Unsupervised Video Summarization To avoid using ground-truth-annotated training data for learning video summarization, most existing unsupervised approaches focus on the principle that a representative  ... 
doi:10.1109/tcsvt.2020.3037883 fatcat:trhxq7kusra7he7eop33lf6tau

Video Summarization through Reinforcement Learning with a 3D Spatio-Temporal U-Net [article]

Tianrui Liu, Qingjie Meng, Jun-Jie Huang, Athanasios Vlontzos, Daniel Rueckert, Bernhard Kainz
2021 arXiv   pre-print
A 3D spatio-temporal U-Net is used to efficiently encode spatio-temporal information of the input videos for downstream reinforcement learning (RL).  ...  In this paper, we introduce the 3DST-UNet-RL framework for video summarization.  ...  the representativeness reward Rrep and diversity reward R div .  ... 
arXiv:2106.10528v1 fatcat:6q3wnioqtrdktkfl5w5n3pjthi

Summarizing Videos using Concentrated Attention and Considering the Uniqueness and Diversity of the Video Frames

Evlampios Apostolidis, Georgios Balaouras, Vasileios Mezaris, Ioannis Patras
2022 Proceedings of the 2022 International Conference on Multimedia Retrieval  
In this work, we describe a new method for unsupervised video summarization.  ...  To overcome limitations of existing unsupervised video summarization approaches, that relate to the unstable training of Generator-Discriminator architectures, the use of RNNs for modeling long-range frames  ...  Session 4B: Captioning and Summarization ICMR '22, June 27-30, 2022, Newark, NJ, USA.  ... 
doi:10.1145/3512527.3531404 fatcat:wj7utepnmvccbeipf4aukg6vpe

How Local is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization [article]

Yandong Li, Liqiang Wang, Tianbao Yang, Boqing Gong
2018 arXiv   pre-print
To tackle these problems, we instead devise a reinforcement learning algorithm for training the proposed model.  ...  The large volume of video content and high viewing frequency demand automatic video summarization algorithms, of which a key property is the capability of modeling diversity.  ...  B.G. would like to thank Trevor Darrell, Charless Fowlkes, Alexander Ihler, Dequan Wang, and Huazhe Xu for the insightful discussions on SeqDPP which inspired this work.  ... 
arXiv:1807.04219v4 fatcat:sewoyvmfxjb5ngxtxrrmr3x7ze

How Local Is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization [chapter]

Yandong Li, Liqiang Wang, Tianbao Yang, Boqing Gong
2018 Lecture Notes in Computer Science  
To tackle these problems, we instead devise a reinforcement learning algorithm for training the proposed model.  ...  The large volume of video content and high viewing frequency demand automatic video summarization algorithms, of which a key property is the capability of modeling diversity.  ...  B.G. would like to thank Trevor Darrell, Charless Fowlkes, Alexander Ihler, Dequan Wang, and Huazhe Xu for the insightful discussions on SeqDPP which inspired this work.  ... 
doi:10.1007/978-3-030-01237-3_10 fatcat:bwqpoy5kxbb5bommka73vn246u
« Previous Showing results 1 — 15 out of 1,798 results