A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
Filters
Guiding Interaction Behaviors for Multi-modal Grounded Language Learning
2017
Proceedings of the First Workshop on Language Grounding for Robotics
Preserved Fulltext
Multi-modal grounded language learning connects language predicates to physical properties of objects in the world. ...
In this work, we gather behavior annotations from humans and demonstrate that these improve language grounding performance by allowing a system to focus on relevant behaviors for words like "white" or ...
Acknowledgments We thank our anonymous reviewers for their time and insights. ...
doi:10.18653/v1/w17-2803
dblp:conf/acl/ThomasonSM17
fatcat:jiiw4ip23vbz5cdsbiq2kp352e
Towards Learning by Interacting
[chapter]
2009
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
For example, in learning situations, the interactive situation has the potential to highlight parts of an action by linguistic and nonlinguistic features and thus to guide the attention of the learner ...
We further propose that such an approach necessarily needs to take three aspects into account: (1) multi-modal integration at all processing levels (2) derivation of top-down strategies from bottom-up ...
new situations and learn qualitatively new concepts as would be necessary for e.g. language learning. ...
doi:10.1007/978-3-642-00616-6_8
fatcat:ooj73bwl2nasrk2gvgtkereqfa
SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Within the Deep Reinforcement Learning (DRL) field, this objective motivated multiple works on embodied language use. ...
Building embodied autonomous agents capable of participating in social interactions with humans is one of the main challenges in AI. ...
[2020] , which is an invitation to focus research on building embodied multi-modal agents suited for human-robot interactions in the real-world. ...
arXiv:2107.00956v3
fatcat:6jyi3eivtfctbl2vl66se2jy3q
Guiding Policies with Language via Meta-Learning
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Behavioral skills or policies for autonomous agents are conventionally learned from reward functions, via reinforcement learning, or from demonstrations, via imitation learning. ...
Our proposed language-guided policy learning algorithm can integrate an instruction and a sequence of corrections to acquire new skills very quickly. ...
DISCUSSION AND FUTURE WORK We presented meta-learning for guided policies with language (GPL), a framework for interactive learning of tasks with in-the-loop language corrections. ...
arXiv:1811.07882v2
fatcat:vbadnxumhvgojgsqoauufdpcq4
M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Moreover, M2Lens identifies frequent and influential multimodal features and supports the multi-faceted exploration of model behaviors from language, acoustic, and visual modalities. ...
M2Lens provides explanations on intra- and inter-modal interactions at the global, subset, and local levels. ...
ACKNOWLEDGMENTS The authors wish to thank anonymous reviewers for their feedback. This research was supported in part by grant FSNH20EG01 under Foshan-HKUST Projects. ...
arXiv:2107.08264v4
fatcat:2zmqziomuveodlru5vqfcvnpta
A FRAMEWORK FOR DEVELOPING CULTURE-BASED MULTI-MODAL MIND GAMES: IMPROVING SOCIAL INTERACTION SKILLS OF AUTISTIC CHILDREN
2015
Jurnal Teknologi
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
One approach that has shown great potential in enhancing social interaction skills among autistic children is the multi-modal mind games approach. ...
Action research method will be adopted since the cyclic nature of the method will provide opportunity for improving existing educational practices for autistic children. ...
Acknowledgement The authors would like to thank the Universiti Tun Hussein Onn Malaysia (UTHM) for funding this research. ...
doi:10.11113/jt.v75.5049
fatcat:xblwvktivbbxjdsyjtzhizboiy
Introduction to the Special Issue on Machine Learning for Multiple Modalities in Interactive Systems and Robots
2014
ACM transactions on interactive intelligent systems (TiiS)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
However, machine learning methods that encompass multiple modalities of an interactive system are still relatively hard to find. ...
This special issue highlights research articles that apply machine learning to robots and other systems that interact with users through more than one modality, such as speech, gestures, and vision. ...
ACKNOWLEDGMENTS We thank the chief editors of ACM TiiS, Anthony Jameson, John Riedl and Krzysztof Gajos, for letting this special issue become a reality and for their high commitment with this journal. ...
doi:10.1145/2670539
fatcat:mpwlonu2yfcnher33owzkl6j6y
Ecological Semantics: Programming Environments for Situated Language Understanding
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Grounded language learning approaches offer the promise of deeper understanding by situating learning in richer, more structured training environments, but are limited in scale to relatively narrow, predefined ...
We further provide actual demonstrations building upon interactive fiction programming languages. ...
The last author is funded by an ERC grant on Natural Language Programming, grant number 677352, for which we are grateful. ...
arXiv:2003.04567v2
fatcat:p7ni2tbhmzdpfieh7jty7qn274
On the Gap between Domestic Robotic Applications and Computational Intelligence
2021
Electronics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
the state-of-the-art research on multi-modal human–machine interaction from various domains, including natural language processing and multi-modal dialogue systems. ...
., deep learning techniques and computational intelligence technologies), robotic products have become available to ordinary household users. ...
Acknowledgments: J.Z. would like to thank his aunt for allowing to stay in her apartment to do the writing during self-quarantine. ...
doi:10.3390/electronics10070793
fatcat:5fwhh2qjpncnhdz7u4mqwathp4
Tailor Versatile Multi-modal Learning for Multi-label Emotion Recognition
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose versaTile multi-modAl learning for multI-labeL emOtion Recognition (TAILOR), aiming to refine multi-modal representations and enhance discriminative capacity of each label. ...
Previous methods mainly focus on projecting multiple modalities into a common latent space and learning an identical representation for all labels, which neglects the diversity of each modality and fails ...
learning for vision-and-language grounding problems. In
Baltrušaitis, T.; Robinson, P.; and Morency, L.-P. 2016. AAAI, volume 34, 11572–11579. ...
arXiv:2201.05834v1
fatcat:ua2xxwh7ezcuvd2o2sbexymbmi
Introduction to Journal of Human-Robot Interaction Special Issue on Haptics in HRI: Cooperation and Communication
2015
Journal of Human-Robot Interaction
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2015; you can also visit the original URL.
The file type is application/pdf.
Coming generations of robots will share physical space with humans, engaging in contact interactions (physical Human Robot Interaction, or pHRI) as they carry out cooperative tasks. ...
This special issue turns a spotlight on the specific roles that crafted haptic interaction can play in cooperation and communication between a human and a robotic partner, from the viewpoints of human ...
Beyond this, we find insights in evaluative methods, modalities of interaction, and use of automated learning techniques based on sensed forces. ...
doi:10.5898/jhri.4.1.maclean
fatcat:yiycbuggs5c4tomf4tdqwfgkwu
A Short Review of Symbol Grounding in Robotic and Intelligent Systems
2013
Künstliche Intelligenz
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
The review covers a number of subtopics, that include, physical symbol grounding, social symbol grounding, symbol grounding for vision systems, anchoring in robotic systems, and learning symbol grounding ...
The focus is in the use of symbol grounding for robotics and intelligent system. ...
Acknowledgements We would like to thank Tony Belpaeme, Fredrik Heintz, Sven Albrecht, Angelo Cangelosi, Paul Vogt, Katerina Pastra and Sverin Lemaignan for their helpful comments to improve the article ...
doi:10.1007/s13218-013-0247-2
fatcat:ghimcniy6na3fk7yqf54typjiu
Two-stage Visual Cues Enhancement Network for Referring Image Segmentation
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Through the two-stage enhancement, our proposed TV-Net enjoys better performances in learning fine-grained matching behaviors between the natural language expression and image, especially when the visual ...
The diverse and flexible expressions as well as complex visual contents in the images raise the RIS model with higher demands for investigating fine-grained matching behaviors between words in expressions ...
Moreover, exploring cross-modal interaction is often formulated as a multi-step progressive process [9, 13, 25, 26] . ...
arXiv:2110.04435v1
fatcat:zt23iztwbjbdhlxaxsnvfmuyei
Human-style interaction with a robot for cooperative learning of scene objects
2005
Proceedings of the 7th international conference on Multimodal interfaces - ICMI '05
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2007; you can also visit the original URL.
The file type is application/pdf.
In research on human-robot interaction the interest is currently shifting from uni-modal dialog systems to multi-modal interaction schemes. ...
To model the dialog we adopt an extended grounding concept with a mechanism to handle multi-modal in-and output where object references are resolved by the interaction with an object attention system ( ...
We define the term human style modalities as multi-modal communication channels that humans are biologically equipped for and (learn to) use from their birth. ...
doi:10.1145/1088463.1088491
dblp:conf/icmi/LiHWFS05
fatcat:lj7cjwsyrndexnaka5ypxh56py
Adaptive Grasp Control through Multi-Modal Interactions for Assistive Prosthetic Devices
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
The approach explored here is to develop algorithms that permit a device to adapt its behavior to the preferences of the operator through interactions with the wearer. ...
Contemporary prosthetic devices for people with transradial amputations or wrist disarticulation vary in complexity, from passive prosthetics to complex devices that are body or electrically driven. ...
The novel aspects of the multi-modal interface for adaptive grasp control are most closely related to the corrections provided through natural language interaction in (Broad et al. 2017), however corrections ...
arXiv:1810.07899v1
fatcat:gbikrjk3jncolozontcmkgsebq
« Previous
Showing results 1 — 15 out of 17,210 results