3 Hits in 1.6 sec

FollowNet: Robot Navigation by Following Natural Language Directions with Deep Reinforcement Learning [article]

Pararth Shah, Marek Fiser, Aleksandra Faust, J. Chase Kew, Dilek Hakkani-Tur
2018 arXiv   pre-print
Understanding and following directions provided by humans can enable robots to navigate effectively in unknown situations.  ...  Deep reinforcement learning (RL) a sparse reward learns simultaneously the state representation, the attention function, and control policies.  ...  We present robots with example directions similar to the one above, and train a deep reinforcement learning (DRL) agent to follow the directions.  ... 
arXiv:1805.06150v1 fatcat:jo5mquk7yjffzgnvdrz3fofihi

Bridging the Gap: Using Deep Acoustic Representations to Learn Grounded Language from Percepts and Raw Speech [article]

Gaoussou Youssouf Kebe, Luke E. Richards, Edward Raff, Francis Ferraro, Cynthia Matuszek
2021 arXiv   pre-print
Learning to understand grounded language, which connects natural language to percepts, is a critical research area.  ...  This will allow interactions in which language about novel tasks and environments is learned from end users, reducing dependence on textual inputs and potentially mitigating the effects of demographic  ...  FollowNet: Robot Navigation by Following Nat- ural Language Directions with Deep Reinforcement Learn- ing. ArXiv, abs/1805.06150. Shridhar, M.; Mittal, D.; and Hsu, D. 2020.  ... 
arXiv:2112.13758v1 fatcat:uuoimxopdrfbhly7sdnu4w7gqi

ArraMon: A Joint Navigation-Assembly Instruction Interpretation Task in Dynamic Environments

Hyounghun Kim, Abhaysinh Zala, Graham Burri, Hao Tan, Mohit Bansal
2020 Findings of the Association for Computational Linguistics: EMNLP 2020   unpublished
During this task, the agent (similar to a PokéMON GO player) is asked to find and collect different target objects one-by-one by navigating based on natural language (English) instructions in a complex  ...  To support this task, we implement a 3D dynamic environment simulator and collect a dataset with human-written navigation and assembling instructions, and the corresponding ground truth trajectories.  ...  This work was supported by NSF Award 1840131, ARO-YIP Award W911NF-18-1-0336, DARPA MCS Grant N66001-19-2-4031, and a Google Foucused Award.  ... 
doi:10.18653/v1/2020.findings-emnlp.348 fatcat:w5gt46tu5rbq5gxdexhg3xsmti