17,210 Hits in 4.4 sec

Guiding Interaction Behaviors for Multi-modal Grounded Language Learning

Jesse Thomason, Jivko Sinapov, Raymond Mooney
2017 Proceedings of the First Workshop on Language Grounding for Robotics  
Multi-modal grounded language learning connects language predicates to physical properties of objects in the world.  ...  In this work, we gather behavior annotations from humans and demonstrate that these improve language grounding performance by allowing a system to focus on relevant behaviors for words like "white" or  ...  Acknowledgments We thank our anonymous reviewers for their time and insights.  ... 
doi:10.18653/v1/w17-2803 dblp:conf/acl/ThomasonSM17 fatcat:jiiw4ip23vbz5cdsbiq2kp352e

Towards Learning by Interacting [chapter]

Britta Wrede, Katharina J. Rohlfing, Marc Hanheide, Gerhard Sagerer
2009 Lecture Notes in Computer Science  
For example, in learning situations, the interactive situation has the potential to highlight parts of an action by linguistic and nonlinguistic features and thus to guide the attention of the learner  ...  We further propose that such an approach necessarily needs to take three aspects into account: (1) multi-modal integration at all processing levels (2) derivation of top-down strategies from bottom-up  ...  new situations and learn qualitatively new concepts as would be necessary for e.g. language learning.  ... 
doi:10.1007/978-3-642-00616-6_8 fatcat:ooj73bwl2nasrk2gvgtkereqfa

SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents [article]

Grgur Kovač, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer
2021 arXiv   pre-print
Within the Deep Reinforcement Learning (DRL) field, this objective motivated multiple works on embodied language use.  ...  Building embodied autonomous agents capable of participating in social interactions with humans is one of the main challenges in AI.  ...  [2020] , which is an invitation to focus research on building embodied multi-modal agents suited for human-robot interactions in the real-world.  ... 
arXiv:2107.00956v3 fatcat:6jyi3eivtfctbl2vl66se2jy3q

Guiding Policies with Language via Meta-Learning [article]

John D. Co-Reyes, Abhishek Gupta, Suvansh Sanjeev, Nick Altieri, Jacob Andreas, John DeNero, Pieter Abbeel, Sergey Levine
2019 arXiv   pre-print
Behavioral skills or policies for autonomous agents are conventionally learned from reward functions, via reinforcement learning, or from demonstrations, via imitation learning.  ...  Our proposed language-guided policy learning algorithm can integrate an instruction and a sequence of corrections to acquire new skills very quickly.  ...  DISCUSSION AND FUTURE WORK We presented meta-learning for guided policies with language (GPL), a framework for interactive learning of tasks with in-the-loop language corrections.  ... 
arXiv:1811.07882v2 fatcat:vbadnxumhvgojgsqoauufdpcq4

M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis [article]

Xingbo Wang, Jianben He, Zhihua Jin, Muqiao Yang, Yong Wang, Huamin Qu
2021 arXiv   pre-print
Moreover, M2Lens identifies frequent and influential multimodal features and supports the multi-faceted exploration of model behaviors from language, acoustic, and visual modalities.  ...  M2Lens provides explanations on intra- and inter-modal interactions at the global, subset, and local levels.  ...  ACKNOWLEDGMENTS The authors wish to thank anonymous reviewers for their feedback. This research was supported in part by grant FSNH20EG01 under Foshan-HKUST Projects.  ... 
arXiv:2107.08264v4 fatcat:2zmqziomuveodlru5vqfcvnpta


Rafizah Mohd Hanifa, Maizam Alias, Ida Aryanie Bahrudin, Miswan Surip, Zuraida Ibrahim, Rosfuzah Roslan
2015 Jurnal Teknologi  
One approach that has shown great potential in enhancing social interaction skills among autistic children is the multi-modal mind games approach.  ...  Action research method will be adopted since the cyclic nature of the method will provide opportunity for improving existing educational practices for autistic children.  ...  Acknowledgement The authors would like to thank the Universiti Tun Hussein Onn Malaysia (UTHM) for funding this research.  ... 
doi:10.11113/jt.v75.5049 fatcat:xblwvktivbbxjdsyjtzhizboiy

Introduction to the Special Issue on Machine Learning for Multiple Modalities in Interactive Systems and Robots

Heriberto Cuayáhuitl, Lutz Frommberger, Nina Dethlefs, Antoine Raux, Mathew Marge, Hendrik Zender
2014 ACM transactions on interactive intelligent systems (TiiS)  
However, machine learning methods that encompass multiple modalities of an interactive system are still relatively hard to find.  ...  This special issue highlights research articles that apply machine learning to robots and other systems that interact with users through more than one modality, such as speech, gestures, and vision.  ...  ACKNOWLEDGMENTS We thank the chief editors of ACM TiiS, Anthony Jameson, John Riedl and Krzysztof Gajos, for letting this special issue become a reality and for their high commitment with this journal.  ... 
doi:10.1145/2670539 fatcat:mpwlonu2yfcnher33owzkl6j6y

Ecological Semantics: Programming Environments for Situated Language Understanding [article]

Ronen Tamari, Gabriel Stanovsky, Dafna Shahaf, Reut Tsarfaty
2020 arXiv   pre-print
Grounded language learning approaches offer the promise of deeper understanding by situating learning in richer, more structured training environments, but are limited in scale to relatively narrow, predefined  ...  We further provide actual demonstrations building upon interactive fiction programming languages.  ...  The last author is funded by an ERC grant on Natural Language Programming, grant number 677352, for which we are grateful.  ... 
arXiv:2003.04567v2 fatcat:p7ni2tbhmzdpfieh7jty7qn274

On the Gap between Domestic Robotic Applications and Computational Intelligence

Junpei Zhong, Chaofan Ling, Angelo Cangelosi, Ahmad Lotfi, Xiaofeng Liu
2021 Electronics  
the state-of-the-art research on multi-modal human–machine interaction from various domains, including natural language processing and multi-modal dialogue systems.  ...  ., deep learning techniques and computational intelligence technologies), robotic products have become available to ordinary household users.  ...  Acknowledgments: J.Z. would like to thank his aunt for allowing to stay in her apartment to do the writing during self-quarantine.  ... 
doi:10.3390/electronics10070793 fatcat:5fwhh2qjpncnhdz7u4mqwathp4

Tailor Versatile Multi-modal Learning for Multi-label Emotion Recognition [article]

Yi Zhang, Mingyuan Chen, Jundong Shen, Chongjun Wang
2022 arXiv   pre-print
In this paper, we propose versaTile multi-modAl learning for multI-labeL emOtion Recognition (TAILOR), aiming to refine multi-modal representations and enhance discriminative capacity of each label.  ...  Previous methods mainly focus on projecting multiple modalities into a common latent space and learning an identical representation for all labels, which neglects the diversity of each modality and fails  ...  learning for vision-and-language grounding problems. In Baltrušaitis, T.; Robinson, P.; and Morency, L.-P. 2016. AAAI, volume 34, 11572–11579.  ... 
arXiv:2201.05834v1 fatcat:ua2xxwh7ezcuvd2o2sbexymbmi

Introduction to Journal of Human-Robot Interaction Special Issue on Haptics in HRI: Cooperation and Communication

Karon E. MacLean, Antonio Frisoli
2015 Journal of Human-Robot Interaction  
Coming generations of robots will share physical space with humans, engaging in contact interactions (physical Human Robot Interaction, or pHRI) as they carry out cooperative tasks.  ...  This special issue turns a spotlight on the specific roles that crafted haptic interaction can play in cooperation and communication between a human and a robotic partner, from the viewpoints of human  ...  Beyond this, we find insights in evaluative methods, modalities of interaction, and use of automated learning techniques based on sensed forces.  ... 
doi:10.5898/jhri.4.1.maclean fatcat:yiycbuggs5c4tomf4tdqwfgkwu

A Short Review of Symbol Grounding in Robotic and Intelligent Systems

Silvia Coradeschi, Amy Loutfi, Britta Wrede
2013 Künstliche Intelligenz  
The review covers a number of subtopics, that include, physical symbol grounding, social symbol grounding, symbol grounding for vision systems, anchoring in robotic systems, and learning symbol grounding  ...  The focus is in the use of symbol grounding for robotics and intelligent system.  ...  Acknowledgements We would like to thank Tony Belpaeme, Fredrik Heintz, Sven Albrecht, Angelo Cangelosi, Paul Vogt, Katerina Pastra and Sverin Lemaignan for their helpful comments to improve the article  ... 
doi:10.1007/s13218-013-0247-2 fatcat:ghimcniy6na3fk7yqf54typjiu

Two-stage Visual Cues Enhancement Network for Referring Image Segmentation [article]

Yang Jiao, Zequn Jie, Weixin Luo, Jingjing Chen, Yu-Gang Jiang, Xiaolin Wei, Lin Ma
2021 arXiv   pre-print
Through the two-stage enhancement, our proposed TV-Net enjoys better performances in learning fine-grained matching behaviors between the natural language expression and image, especially when the visual  ...  The diverse and flexible expressions as well as complex visual contents in the images raise the RIS model with higher demands for investigating fine-grained matching behaviors between words in expressions  ...  Moreover, exploring cross-modal interaction is often formulated as a multi-step progressive process [9, 13, 25, 26] .  ... 
arXiv:2110.04435v1 fatcat:zt23iztwbjbdhlxaxsnvfmuyei

Human-style interaction with a robot for cooperative learning of scene objects

Shuyin Li, Axel Haasch, Britta Wrede, Jannik Fritsch, Gerhard Sagerer
2005 Proceedings of the 7th international conference on Multimodal interfaces - ICMI '05  
In research on human-robot interaction the interest is currently shifting from uni-modal dialog systems to multi-modal interaction schemes.  ...  To model the dialog we adopt an extended grounding concept with a mechanism to handle multi-modal in-and output where object references are resolved by the interaction with an object attention system (  ...  We define the term human style modalities as multi-modal communication channels that humans are biologically equipped for and (learn to) use from their birth.  ... 
doi:10.1145/1088463.1088491 dblp:conf/icmi/LiHWFS05 fatcat:lj7cjwsyrndexnaka5ypxh56py

Adaptive Grasp Control through Multi-Modal Interactions for Assistive Prosthetic Devices [article]

Michelle Esponda, Thomas M. Howard
2018 arXiv   pre-print
The approach explored here is to develop algorithms that permit a device to adapt its behavior to the preferences of the operator through interactions with the wearer.  ...  Contemporary prosthetic devices for people with transradial amputations or wrist disarticulation vary in complexity, from passive prosthetics to complex devices that are body or electrically driven.  ...  The novel aspects of the multi-modal interface for adaptive grasp control are most closely related to the corrections provided through natural language interaction in (Broad et al. 2017), however corrections  ... 
arXiv:1810.07899v1 fatcat:gbikrjk3jncolozontcmkgsebq
« Previous Showing results 1 — 15 out of 17,210 results