Filters








27 Hits in 3.5 sec

"Hey, vitrivr!" – A Multimodal UI for Video Retrieval [chapter]

Prateek Goel, Ivan Giangreco, Luca Rossetto, Claudiu Tănase, Heiko Schuldt
2017 Lecture Notes in Computer Science  
In this paper, we present a multimodal web-based user interface for the vitrivr system. vitrivr is a modern, open-source video retrieval system for searching in large collections of video using a great  ...  With the multimodal user interface, prospective users benefit from being able to naturally interact with the vitrivr system by using spoken commands and also by applying multimodal commands which combine  ...  Acknowledgments This work was partly funded by the Swiss National Science Foundation (SNSF) in the context of the Chist-Era program IMOTION (contract no. 20CH21 151571), and the Google Summer of Code 2016  ... 
doi:10.1007/978-3-319-56608-5_75 fatcat:f5lymchtmvhf7bmjjea7bxr2zm

Multimodal Multimedia Retrieval with vitrivr

Ralph Gasser, Luca Rossetto, Heiko Schuldt
2019 Proceedings of the 2019 on International Conference on Multimedia Retrieval - ICMR '19  
The steady growth of multimedia collections -both in terms of size and heterogeneity -necessitates systems that are able to conjointly deal with several types of media as well as large volumes of data.  ...  To the best of our knowledge, the full vitrivr stack is unique in that it seamlessly integrates support for four different types of media, namely images, audio, videos, and 3D models.  ...  ACKNOWLEDGEMENTS This work was partly supported by the Swiss National Science Foundation, project IMOTION (20CH21_151571).  ... 
doi:10.1145/3323873.3326921 dblp:conf/mir/GasserRS19 fatcat:cr7iqzlznneptha3ik4whfomkq

Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis

Zhongkai Sun, Prathusha Sarma, William Sethares, Yingyu Liang
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
Given that the text, audio, and video are describing the same utterance in different ways, we hypothesize that the multimodal sentiment analysis and emotion recognition can be improved by learning (hidden  ...  This paper proposes a novel model, the Interaction Canonical Correlation Network (ICCN), to learn such multimodal embeddings.  ...  • Video Features: Facet(iMotions 2017) has been used for extracting facial expression features such as action units and face pose.  ... 
doi:10.1609/aaai.v34i05.6431 fatcat:ybwaoags4bhtppi4nxwehgtkpq

The Impact of Surrounding Road Objects and Conditions on Drivers Abrupt Heart Rate Changes [article]

Arash Tavakoli, Arsalan Heydarian
2022 arXiv   pre-print
In this paper, based on a naturalistic multimodal driving dataset, we analyze the visual scenes of driving in the vicinity of abrupt increases in drivers HR for the presence of certain stress-inducing  ...  These studies show that certain road objects such as big vehicles might be associated with higher stress levels based on drivers subjective stress measures.  ...  A sample of the road objects detected by this method on our videos is shown in Figure 1 -D . This algorithm is applied to all the outside video frames that were retrieved from the previous section.  ... 
arXiv:2205.11014v1 fatcat:w2a3auyvvzb7fbjp6pnc2u4bsy

Audiovisual Science Communication on TV and YouTube. How Recipients Understand and Evaluate Science Videos

Bettina Boy, Hans-Jürgen Bucher, Katharina Christ
2020 Frontiers in Communication  
With the emergence of the Internet, social media and video platforms are gaining considerable influence on the traditional media landscape in general and on science communication in particular.  ...  Therefore, this study combines a discourse analysis with the aim to create a typology of YouTube videosthe independent variables—and an audience study for investigating knowledge transfer—the dependent  ...  The entropy evaluation of gaze data was supported by Kerstin Wolf of iMotions and Stephan Hill, computer scientist at the University of Koblenz.  ... 
doi:10.3389/fcomm.2020.608620 fatcat:5nvuvyn44jht5de75bbo472ojy

Development of an Effective Bootleg Videos Retrieval System as a Part of Content-Based Video Search Engine

Ahmad Sedky Adly, Islam Hegazy, Taha Elarif, M. S. Abdelwahab
2022 International Scientific Journal of Computing  
The retrieval system was evaluated using more than 200 non-semantic-based video queries evaluating both normal and bootleg videos, with retrieval precision for normal videos of 97.9% and retrieval recall  ...  Bootleg videos retrieval precision scored 99.2% and retrieval recall was of 96.7% combined by the F1 measure to be 97.9%.  ...  CONTENT-BASED VIDEO RETRIEVAL SYSTEM The main target of building a content-based video search engine is to have a video retrieval system with the objective of retrieving users' queries with the most relevant  ... 
doi:10.47839/ijc.21.2.2590 fatcat:ide3dr3aerarxczzi72laidtdm

HARMONY: A Human-centered Multimodal Driving Study in the Wild

Arash Tavakoli, Shashwat Kumar, Xiang Guo, Vahid Balali, Mehdi Boukhechba, Arsalan Heydarian
2021 IEEE Access  
OpenPose detects skeleton joints in each video. Finally, using the Affectiva module on iMotion software, multiple emotional metrics are retrieved, including 2D and 1D emotional measures.  ...  By using the file information on each video, we retrieve the name of the video, participant ID, duration of each video, start time, and end time.  ... 
doi:10.1109/access.2021.3056007 fatcat:qczxgrbxsfdwvfawemlhyclspi

A review of affective computing: From unimodal analysis to multimodal fusion

Soujanya Poria, Erik Cambria, Rajiv Bajpai, Amir Hussain
2017 Information Fusion  
With the proliferation of videos posted online (e.g., on YouTube, Facebook, Twitter) for product reviews, movie reviews, political views, and more, affective computing research has increasingly evolved  ...  As part of this review, we carry out an extensive study of different categories of state-of-the-art fusion techniques, followed by a critical analysis of potential performance improvements with multimodal  ...  The multimodal system was also integrated with GAVAM head tracker, CLM-Z face tracker, SHORE face detector, and more.  ... 
doi:10.1016/j.inffus.2017.02.003 fatcat:ytebhjxlz5bvxcdghg4wxbvr6a

The Effect of Emotions on Self-Regulated-Learning (SRL) and Story Comprehension in Emerging Readers

Stephanie Buono, Ana Zdravkovic, Milan Lazic, Earl Woodruff
2020 Frontiers in Education  
In the early-elementary years, emerging readers experience many emotions due to the high value that is placed on the development of reading skills.  ...  The participants were 150 emerging readers (M age = 8, SD = 10.63, 56% girls).  ...  Video recordings of the participants facial expressions were collected, and facial expression recognition software coded expressions of emotion (iMotions, 2018).  ... 
doi:10.3389/feduc.2020.588043 fatcat:qvjdx64dlrdanp4psv57b5dnt4

JAKE® Multimodal Data Capture System: Insights from an Observational Study of Autism Spectrum Disorder

Seth L. Ness, Nikolay V. Manyakov, Abigail Bangerter, David Lewin, Shyla Jagannatha, Matthew Boice, Andrew Skalkin, Geraldine Dawson, Yvette M. Janvier, Matthew S. Goodwin, Robert Hendren, Bennett Leventhal (+3 others)
2017 Frontiers in Neuroscience  
Methods: A prospective, observational study of 29 children and adolescents with ASD using the JAKE system was conducted at three sites in the United States.  ...  Conclusion: The results of this study, with the described changes, demonstrate that the JAKE system is a viable, useful, and safe platform for use in clinical trials of ASD, justifying larger validation  ...  The JAKE system was successful in gathering responses to the ABI (Bangerter et al., 2017) .  ... 
doi:10.3389/fnins.2017.00517 pmid:29018317 pmcid:PMC5623040 fatcat:dr74ul6exja7tgnb2ry4nms4hu

Bringing Emotion Recognition Out of the Lab into Real Life: Recent Advances in Sensors and Machine Learning

Stanisław Saganowski
2022 Electronics  
The review is concluded with a debate on what challenges need to be overcome in the domain in the near future.  ...  assisting kids with autism, monitoring the elderly, and general improvement of well-being.  ...  Existing Systems The iMotions is a platform with the broadest spectrum of services targeted at affective research and analysis for scientific and industry clients [79] .  ... 
doi:10.3390/electronics11030496 fatcat:pagrnyshp5fq7nkcdcqd2gzdbm

From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video [article]

Junwei Liang
2021 arXiv   pre-print
With the advancement in computer vision deep learning, systems now are able to analyze an unprecedented amount of rich visual information from videos to enable applications such as autonomous driving,  ...  To enable optimal future human behavioral forecasting, it is crucial for the system to be able to detect and analyze human activities as well as scene semantics, passing informative features to the subsequent  ...  FastImage [69] is a video retrieval method that utilizes web images from search engine to match to the video with re-ranking. WELL-MM is the proposed method.  ... 
arXiv:2011.10670v3 fatcat:mlom5zqk6jdvjndcsfwimpj7xu

Wearables for Engagement Detection in Learning Environments: A Review

Maritza Bustos-López, Nicandro Cruz-Ramírez, Alejandro Guerra-Hernández, Laura Nely Sánchez-Morales, Nancy Aracely Cruz-Ramos, Giner Alor-Hernández
2022 Biosensors  
This work is a review of current student engagement detection initiatives in the educational domain.  ...  The review highlights existing commercial and non-commercial wearables for student engagement monitoring and identifies key physiological signals involved in engagement detection.  ...  Patch The patch detects multimodal biosignals, measures skin temperature with a sensitivity of 0.31 Ω/ • C, skin conductance with a sensitivity of 0.28 µV/0.02 µS, and pulse wave with a response time of  ... 
doi:10.3390/bios12070509 pmid:35884312 pmcid:PMC9312492 fatcat:vfnvxlo6vjgv3m5d3lx5rxdi2m

Volume 1 of the Proceedings of the joint meeting of the 12th International Conference on Measuring Behavior and the 6th Seminar on Behavioral Methods to be held in Krakow, Poland, October 15-18 2021 [article]

Andrew Spink, Jarosław Barski, Anne-Marie Brouwer, Gernot Riedel, Annesha Sil
2020 figshare.com  
This is volume 1 of the Measuring Behavior 2020-21 Conference. Volume 2 will follow when the conference takes place in October 2021. www.measuringbehavior.org.  ...  Any opinions, finding, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the United States Air Force.  ...  Acknowledgments The authors would like to thank: Neil Thomas for providing test data, Eric Lo for providing additional guidance on the algorithm, Allison Busa for initial experiments on machine learning  ... 
doi:10.6084/m9.figshare.13013717.v1 fatcat:a5boldd3xjd5pgzkp5crt2mq5m

Volume 1 of the Proceedings of the joint 12th International Conference on Methods and Techniques in Behavioral Research and 6th Seminar on Behavioral Methods to be held in Krakow, Poland, October 15-18, 2021 [article]

Andrew Spink, Jarosław Barski, Anne-Marie Brouwer, Gernot Riedel, Annesha Sil
2020 figshare.com  
This is volume 1 of the Measuring Behavior 2020-21 Conference. Volume 2 will follow when the conference takes place in October 2021. www.measuringbehavior.org.  ...  Any opinions, finding, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the United States Air Force.  ...  Acknowledgments The authors would like to thank: Neil Thomas for providing test data, Eric Lo for providing additional guidance on the algorithm, Allison Busa for initial experiments on machine learning  ... 
doi:10.6084/m9.figshare.13013717.v3 fatcat:vurhypp455epdf6t5dkecktvsa
« Previous Showing results 1 — 15 out of 27 results