Filters








17 Hits in 4.0 sec

RoboTHOR: An Open Simulation-to-Real Embodied AI Platform [article]

Matt Deitke, Winson Han, Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mottaghi, Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, Luca Weihs, Mark Yatskar (+1 others)
2020 arXiv   pre-print
RoboTHOR offers a framework of simulated environments paired with physical counterparts to systematically explore and overcome the challenges of simulation-to-real transfer, and a platform where researchers  ...  The creation of a comparable ecosystem for simulation-to-real embodied AI presents many challenges: (1) the inherently interactive nature of the problem, (2) the need for tight alignments between real  ...  This work is in part supported by NSF IIS 1652052, IIS 17303166, DARPA N66001-19-2-4031, 67102239 and gifts from Allen Institute for AI.  ... 
arXiv:2004.06799v1 fatcat:4kl653sotvhxxer44w2ogwgn4m

RoboTHOR: An Open Simulation-to-Real Embodied AI Platform

Matt Deitke, Winson Han, Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mottaghi, Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, Luca Weihs, Mark Yatskar (+1 others)
2020 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
We present ROBOTHOR, a platform to develop and test embodied AI agents with corresponding environments in simulation and the physical world.  ...  The complexity of environments in ROBOTHOR along with disparities in appearance and control dynamics between simulation and reality pose new challenges and open many avenues for further research.  ...  A key promise of ROBOTHOR is to serve as an open and accessible benchmarking platform to stimulate reproducible research in embodied AI.  ... 
doi:10.1109/cvpr42600.2020.00323 dblp:conf/cvpr/DeitkeHHKKMSSVW20 fatcat:4hfidirz2ndc7ouo4viqjz3jeq

ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects [article]

Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, Erik Wijmans
2020 arXiv   pre-print
Finally, we provide a detailed description of the instantiation of these recommendations in challenges organized at the Embodied AI workshop at CVPR 2020 http://embodied-ai.org .  ...  In particular, the agent is initialized at a random location and pose in an environment and asked to find an instance of an object category, e.g., find a chair, by navigating to it.  ...  Conclusion Our goal is to provide a common task framework for the task of visual navigation towards objects (ObjectNav).  ... 
arXiv:2006.13171v2 fatcat:3rkub2eqzbh5vfusvymmyctw6m

AllenAct: A Framework for Embodied AI Research [article]

Luca Weihs, Jordi Salvador, Klemen Kotar, Unnat Jain, Kuo-Hao Zeng, Roozbeh Mottaghi, Aniruddha Kembhavi
2020 arXiv   pre-print
We hope that our framework makes Embodied AI more accessible and encourages new researchers to join this exciting area. The framework can be accessed at: https://allenact.org/  ...  The domain of Embodied AI, in which agents learn to complete tasks through interaction with their environment from egocentric observations, has experienced substantial growth with the advent of deep reinforcement  ...  also physical counterparts to simulation environments (RoboTHOR [10] and iGibson [11] ) to enable research in simulation-to-real transfer.  ... 
arXiv:2008.12760v1 fatcat:a5lcfyqojnb4xgejae6i3wpaae

ProcTHOR: Large-Scale Embodied AI Using Procedural Generation [article]

Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Eric Kolve, Ali Farhadi, Aniruddha Kembhavi, Roozbeh Mottaghi
2022 arXiv   pre-print
This work presents a platform to enable similar success stories in Embodied AI. We propose ProcTHOR, a framework for procedural generation of Embodied AI environments.  ...  ProcTHOR enables us to sample arbitrarily large datasets of diverse, interactive, customizable, and performant virtual environments to train and evaluate embodied agents across navigation, interaction,  ...  Acknowledgements We would like to thank the teams behind the open-source packages used in this project, including AI2-THOR [63] , AllenAct [116] , Habitat [101] , Datasets [68] , NumPy [45] , PyTorch  ... 
arXiv:2206.06994v1 fatcat:2rzkcdxttjgzpkvi2e3ovwhdn4

A Survey of Embodied AI: From Simulators to Research Tasks [article]

Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, Cheston Tan
2022 arXiv   pre-print
This paper aims to provide an encyclopedic survey for the field of embodied AI, from its simulators to its research.  ...  Consequently, there has been substantial growth in the demand for embodied AI simulators to support various embodied AI research tasks.  ...  This paucity of world-based scene simulators is the bottleneck to simulation-to-real tasks for embodied AI agents, which further hinders the transferability of embodied AI research into real-world deployment  ... 
arXiv:2103.04918v8 fatcat:2zu4klcchbhnvmjej5ry3emu4u

BenchBot: Evaluating Robotics Research in Photorealistic 3D Simulation and on Real Robots [article]

Ben Talbot, David Hall, Haoyang Zhang, Suman Raj Bista, Rohan Smith, Feras Dayoub, Niko Sünderhauf
2020 arXiv   pre-print
BenchBot provides a simple interface to the sensorimotor capabilities of a robot when solving robotics research problems; an interface that is consistent regardless of whether the target platform is simulated  ...  We introduce BenchBot, a novel software suite for benchmarking the performance of robotics research across both photorealistic 3D simulations and real robot platforms.  ...  An interesting approach being newly adopted, is to provide remote access to robot platforms. This is present in challenges like RoboThor [11] , iGibson [12] and the Real Robot Challenge 2 .  ... 
arXiv:2008.00635v1 fatcat:6j7idhvrlveqdcwidtpnms6tya

Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI [article]

Santhosh K. Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X. Chang, Manolis Savva, Yili Zhao (+1 others)
2021 arXiv   pre-print
The increased scale, fidelity, and diversity of HM3D directly impacts the performance of embodied AI agents trained using it.  ...  When compared to existing photorealistic 3D datasets such as Replica, MP3D, Gibson, and ScanNet, images rendered from HM3D have 20 - 85% higher visual fidelity w.r.t. counterpart images captured with real  ...  Experiments A popular downstream application for large-scale 3D reconstruction datasets has been to use them with 3D simulation platforms [6, 14, 19] to study embodied AI tasks such as visual navigation  ... 
arXiv:2109.08238v1 fatcat:nmgnqu3fercwvjddmj2blulx5y

Deep Learning for Embodied Vision Navigation: A Survey [article]

Fengda Zhu, Yi Zhu, Vincent CS Lee, Xiaodan Liang, Xiaojun Chang
2021 arXiv   pre-print
Building such an agent that observes, thinks, and acts is a key to real intelligence.  ...  "Embodied visual navigation" problem requires an agent to navigate in a 3D environment mainly rely on its first-person observation.  ...  Embodied Simulators An embodied simulator provides an interface for an agent to interact with the environment. We compare different features of the existing simulators in Tab. 2.  ... 
arXiv:2108.04097v4 fatcat:46p2p3zlivabbn7dvowkyccufe

A Survey on Human-aware Robot Navigation [article]

Ronja Möller, Antonino Furnari, Sebastiano Battiato, Aki Härmä, Giovanni Maria Farinella
2021 arXiv   pre-print
Physical manifestations of those systems on the other hand, in the form of embodied agents or robots, have so far been used only for specific applications and are often limited to functional roles (e.g  ...  This paper is concerned with the navigation aspect of a socially-compliant robot and provides a survey of existing solutions for the relevant areas of research as well as an outlook on possible future  ...  Although Pose Estimation is wellexplored to the point that robust cross-platform solutions such as Open-Pose 48 [ [223] ).  ... 
arXiv:2106.11650v1 fatcat:rhutta3sqrhk3po4hmi77h33qy

Core Challenges in Embodied Vision-Language Planning [article]

Jonathan Francis, Nariaki Kitamura, Felix Labelle, Xiaopeng Lu, Ingrid Navarro, Jean Oh
2022 arXiv   pre-print
, and Embodied AI.  ...  We propose a taxonomy to unify these tasks and provide an in-depth analysis and comparison of the new and current algorithmic approaches, metrics, simulated environments, as well as the datasets used for  ...  All authors contributed equally to the manuscript; author list in alphabetical order. Professor Jean Oh is the corresponding author, reachable at jeanoh@cmu.edu.  ... 
arXiv:2106.13948v4 fatcat:esrtfxpun5ae5kaydjnymf3v6u

Core Challenges in Embodied Vision-Language Planning

Jonathan Francis, Nariaki Kitamura, Felix Labelle, Xiaopeng Lu, Ingrid Navarro, Jean Oh
2022 The Journal of Artificial Intelligence Research  
, and Embodied AI.  ...  We propose a taxonomy to unify these tasks and provide an in-depth analysis and comparison of the new and current algorithmic approaches, metrics, simulated environments, as well as the datasets used for  ...  All authors contributed equally to the manuscript; author list in alphabetical order. Professor Jean Oh is the corresponding author, reachable at jeanoh@cmu.edu.  ... 
doi:10.1613/jair.1.13646 fatcat:rmgy6whqefeuvpvte7ultyyjlq

Reinforcement Learning in Practice: Opportunities and Challenges [article]

Yuxi Li
2022 arXiv   pre-print
In this article, we first give a brief introduction to reinforcement learning (RL), and its relationship with deep learning, machine learning and AI.  ...  Then we discuss challenges, in particular, 1) foundation, 2) representation, 3) reward, 4) exploration, 5) model, simulation, planning, and benchmarks, 6) off-policy/offline learning, 7) learning to learn  ...  Deitke et al. propose RoboTHOR, an open sim-to-real embodied AI platform. Gondal et al. (2019) propose a distanglement dataset to study the sim-to-real transfer of inductive bias.  ... 
arXiv:2202.11296v2 fatcat:xdtsmme22rfpfn6rgfotcspnhy

POMP++: Pomcp-based Active Visual Search in unknown indoor environments [article]

Francesco Giuliari, Alberto Castellini, Riccardo Berra, Alessio Del Bue, Alessandro Farinelli, Marco Cristani, Francesco Setti, Yiming Wang
2021 arXiv   pre-print
We evaluate our method on two public benchmark datasets, AVD that is acquired by real robotic platforms and Habitat ObjectNav that is rendered from real 3D scene scans, achieving the best success rate  ...  with an improvement of >10% over the state-of-the-art methods.  ...  Evaluation on Habitat ObjectNav Habitat [27] is a simulation platform designed for embodied AI, where robots can move and observe within realistic 3D environments, e.g. 3D scans of real large scenes  ... 
arXiv:2107.00914v2 fatcat:u2edvnoapfgppcewu7xc5fi3cu

Table of Contents

2020 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
: An Open Simulation-to-Real Embodied AI Platform 3161 Matt Deitke (University of Washington), Winson Han (PRIOR @ Allen Institute for AI), Alvaro Herrasti (PRIOR @ Allen Institute for AI), Aniruddha  ...  Lang (nuTonomy: An APTIV Company), Bassam Helou (nuTonomy: An APTIV Company), and Oscar Beijbom (nuTonomy: An APTIV Company) Rethinking Zero-Shot Video Classification: End-to-End Training for Realistic  ... 
doi:10.1109/cvpr42600.2020.00004 fatcat:c7els2kee5cq7lh6cemeqhdcoa
« Previous Showing results 1 — 15 out of 17 results