734 Hits in 3.5 sec

Video Relationship Reasoning using Gated Spatio-Temporal Energy Graph [article]

Yao-Hung Hubert Tsai and Santosh Divvala and Louis-Philippe Morency and Ruslan Salakhutdinov and Ali Farhadi
2019 arXiv   pre-print
Due to their temporal nature, videos enable us to model and reason about a more comprehensive set of visual relationships, such as those requiring multiple (temporal) observations (e.g., 'man, lift up,  ...  In this paper, we construct a Conditional Random Field on a fully-connected spatio-temporal graph that exploits the statistical dependency between relational entities spatially and temporally.  ...  Conclusion In this paper, we have presented a Gated Spatio-Temporal Energy Graph (GSTEG) model for the task of visual relationship reasoning in videos.  ... 
arXiv:1903.10547v2 fatcat:bdysya3i6nc55pgip4qs23ed7q

Video Relationship Reasoning Using Gated Spatio-Temporal Energy Graph

Yao-Hung Hubert Tsai, Santosh Divvala, Louis-Philippe Morency, Ruslan Salakhutdinov, Ali Farhadi
2019 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
Due to their temporal nature, videos enable us to model and reason about a more comprehensive set of visual relationships, such as those requiring multiple (temporal) observations (e.g., {man, lift up,  ...  We introduce a novel gated energy function parametrization that learns adaptive relations conditioned on visual observations.  ...  Conclusion In this paper, we have presented a Gated Spatio-Temporal Energy Graph (GSTEG) model for the task of visual relationship reasoning in videos.  ... 
doi:10.1109/cvpr.2019.01067 dblp:conf/cvpr/TsaiDMSF19 fatcat:phfdoxwhyvegzmixrkg2iua5o4

Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentences [article]

Zhu Zhang, Zhou Zhao, Yang Zhao, Qi Wang, Huasheng Liu, Lianli Gao
2020 arXiv   pre-print
Thus, we then propose a novel Spatio-Temporal Graph Reasoning Network (STGRN) for this task.  ...  First, we build a spatio-temporal region graph to capture the region relationships with temporal object dynamics, which involves the implicit and explicit spatial subgraphs in each frame and the temporal  ...  To tackle above problems, we propose a novel Spatio-Temporal Graph Reasoning Network (STGRN) to capture region relationships with temporal object dynamics and directly localize the spatio-temporal tubes  ... 
arXiv:2001.06891v3 fatcat:df3uigkdrzbxdnfcze3ydpicpi

Visual Relation Grounding in Videos [article]

Junbin Xiao, Xindi Shang, Xun Yang, Sheng Tang, Tat-Seng Chua
2020 arXiv   pre-print
To ground the relations, we tackle the challenges by collaboratively optimizing two sequences of regions over a constructed hierarchical spatio-temporal region graph through relation attending and reconstruction  ...  The task aims at spatio-temporally localizing the given relations in the form of subject-predicate-object in the videos, so as to provide supportive visual facts for other high-level video-language tasks  ...  Tsai, Y.H.H., Divvala, S., Morency, L.P., Salakhutdinov, R., Farhadi, A.: Video relationship reasoning using gated spatio-temporal energy graph.  ... 
arXiv:2007.08814v2 fatcat:24nbfoj3kbcvtpjcqm22ndsqj4

An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions

Mehrsan Javan Roshtkhari, Martin D. Levine
2013 Computer Vision and Image Understanding  
The spatio-temporal compositions of video volumes are modeled using a probabilistic framework, which calculates their likelihood of being normal in the video.  ...  These salient events are obtained in real-time by detecting anomalous spatio-temporal regions in a densely sampled video.  ...  Given the above assumptions, an ensemble of volumes can be represented as a graph of codewords and their spatio-temporal relationship, as shown in Fig. 6b .  ... 
doi:10.1016/j.cviu.2013.06.007 fatcat:k7o23j7unneixguixfqc7iyhgy

stagNet: An Attentive Semantic RNN for Group Activity and Individual Action Recognition

Mengshi Qi, Yunhong Wang, Jie Qin, Annan Li, Jiebo Luo, Luc Van Gool
2019 IEEE transactions on circuits and systems for video technology (Print)  
In a complex dynamic scene, a crucial yet challenging issue is how to better model the spatio-temporal contextual information and inter-person relationship.  ...  In the paper, we present a novel attentive semantic recurrent neural network (RNN), namely stagNet, for understanding group activities and individual actions in videos, by combining the spatio-temporal  ...  Group activity understanding via a semantic graph. Using the semantic mapping, individual actions and group activity are shown on the semantic graph, which reasons inter-group relationship.  ... 
doi:10.1109/tcsvt.2019.2894161 fatcat:wcjvyo3wgfbsfcew4x62sw6cfi

A Comprehensive Review of Group Activity Recognition in Videos

Li-Fang Wu, Qi Wang, Meng Jian, Yu Qiao, Bo-Xuan Zhao
2021 International Journal of Automation and Computing  
First, we provide a summary and comparison of 11 GAR video datasets in this field.  ...  In this paper, we give a comprehensive overview of the advances in group activity recognition in videos during the past 20 years.  ...  In [45] , multiple tracks in multi-cameras are used to extract spatio-temporal features of individuals.  ... 
doi:10.1007/s11633-020-1258-8 fatcat:ycka4thcy5a6vghpenpthtrndi

Generative Adversarial Networks for Spatio-temporal Data: A Survey [article]

Nan Gao, Hao Xue, Wei Shao, Sichen Zhao, Kyle Kai Qin, Arian Prabowo, Mohammad Saiedur Rahaman, Flora D. Salim
2021 arXiv   pre-print
We summarise the application of popular GAN architectures for spatio-temporal data and the common practices for evaluating the performance of spatio-temporal applications with GANs.  ...  In this paper, we have conducted a comprehensive review of the recent developments of GANs for spatio-temporal data.  ...  Spatio-temporal Graph. Spatio-temporal graph structure provides the representation of the relations between different nodes in different time.  ... 
arXiv:2008.08903v3 fatcat:pbhxbfgw65bodksjdmwazwo4dq

Logic for electromagnetic field patterns [article]

G.a. Kouzaev
2008 arXiv   pre-print
Two gates of this sort are simulated. A short review on semiconductor hardware for this type of spatial digital processing and computing is given.  ...  Such an effect can be modeled by topologically modulated spatio-time electromagnetic signals which theory is proposed in this paper.  ...  Electronic technology uses the amplification and gating, and such circuitry for spatio-time signals was developed in [20, 28, 29] .  ... 
arXiv:0805.4600v1 fatcat:pdubr5w7hzb5ppfwkvyp3k23ty

RGB-D Data-Based Action Recognition: A Review

Muhammad Bilal Shaikh, Douglas Chai
2021 Sensors  
Producer ST-GCN Spatio-Temporal Graph Convolutional Networks SVM Scalar Vector Machines TSM Temporal Shift Module VATN Video Action Transformer Network AME Accumulation of Motion Energy AVA Atomic Visual  ...  [105] have embedded temporal information with dense motion trajectories to learn actions. Yan et al. [71] have modeled relationships between graphs and joints by using a graph-oriented CNN.  ... 
doi:10.3390/s21124246 fatcat:7dvocdy63rckne5yunhfsnr4p4

Deep Fully Connected Model For Collective Activity Recognition

Jichao Liu, Chuanxu Wang, Yuting Gong, Xue Hao
2019 IEEE Access  
In this paper, we propose a deep fully-connected model for group recognition, first we use the spatial-temporal model based on convolution neural network (CNN) and long short-term memory networks (LSTM  ...  Group activity recognition is a challenging task because there is an exponentially large number of semantic and geometrical relationships among individuals.  ...  SPATIAL-TEMPORAL MODEL BASED ON CNN AND LSTM NETWORK We fine-tuned the Alexnet model which was trained by the ImageNet dataset, video image is used as input, and LSTM network is used to describe the temporal  ... 
doi:10.1109/access.2019.2929684 fatcat:zoo47o5o2zbp3jslmczfbd774m

Spatio-Temporal Laplacian Pyramid Coding for Action Recognition

Ling Shao, Xiantong Zhen, Dacheng Tao, Xuelong Li
2014 IEEE Transactions on Cybernetics  
In contrast to sparse representations based on detected local interest points, STLPC regards a video sequence as a whole with spatio-temporal features directly extracted from it, which prevents the loss  ...  Since the convolving and pooling are performed spatio-temporally, the coding model can capture structural and motion information simultaneously and provide an informative representation of actions.  ...  provide a useful and reasonably accurate description of most spatial aspects of simple receptive fields.  ... 
doi:10.1109/tcyb.2013.2273174 pmid:23912503 fatcat:jjqjkgwdcnhsdm4adfwgppcbpy

Human Action Recognition from Various Data Modalities: A Review [article]

Zehua Sun, Qiuhong Ke, Hossein Rahmani, Mohammed Bennamoun, Gang Wang, Jun Liu
2021 arXiv   pre-print
of useful yet distinct information and have various advantages depending on the application scenarios.  ...  Consequently, lots of existing works have attempted to investigate different types of approaches for HAR using various modalities.  ...  spatio-temporal feature gating to enhance HAR.  ... 
arXiv:2012.11866v4 fatcat:twjnaur2jzahznci6clkadylay

Facial Landmark-Based Emotion Recognition via Directed Graph Neural Network

Quang Tran Ngoc, Seunghyun Lee, Byung Cheol Song
2020 Electronics  
Also, in order to prevent the vanishing gradient problem, we further utilized a stable form of a temporal block in the graph framework.  ...  By using graph neural networks, we could capture emotional information through faces' inherent properties, like geometrical and temporary information.  ...  [17] proposed a novel spatio-temporal graph routing (STGR) to use skeleton data for the action recognition task.  ... 
doi:10.3390/electronics9050764 fatcat:bp73dqwbdrddxmnjofik6dnoxu

Identifying Most Walkable Direction for Navigation in an Outdoor Environment [article]

Sachin Mehta, Hannaneh Hajishirzi, Linda Shapiro
2017 arXiv   pre-print
in the scene using a spatio-temporal graph.  ...  Our approach extracts semantically rich contextual information from the scene using a custom encoder-decoder architecture for semantic segmentation and models the spatial and temporal behavior of objects  ...  Spatio-temporal graphs have been used in applications, such as video summarization [34] , driver assistance [71, 28] and activity recognition [6, 31, 29] , for reasoning about spatial and temporal  ... 
arXiv:1711.08040v2 fatcat:65dxqskvdbaltlgx2sptnsdije
« Previous Showing results 1 — 15 out of 734 results