295 Hits in 5.6 sec

Manipulation Planning for Object Re-Orientation Based on Semantic Segmentation Keypoint Detection

Ching-Chang Wong, Li-Yu Yeh, Chih-Cheng Liu, Chi-Yi Tsai, Hisasuki Aoyama
2021 Sensors  
In this paper, a manipulation planning method for object re-orientation based on semantic segmentation keypoint detection is proposed for robot manipulator which is able to detect and re-orientate the  ...  There are two main parts: (1) 3D keypoint detection system; and (2) manipulation planning system for object re-orientation.  ...  Conclusions An object re-orientation planning method based on 3D keypoint detection is proposed for the robot manipulator so that it can re-orientate an object from an arbitrary pose to a specified position  ... 
doi:10.3390/s21072280 pmid:33805211 fatcat:zhysgbmfkjhqhijhl4anrv4654

kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation [article]

Lucas Manuelli, Wei Gao, Peter Florence, Russ Tedrake
2019 arXiv   pre-print
Using this formulation, we factor the manipulation policy into instance segmentation, 3D keypoint detection, optimization-based robot action planning and local dense-geometry-based action execution.  ...  Hence we propose a novel formulation of category-level manipulation that uses semantic 3D keypoints as the object representation.  ...  Using this formulation, we con- tribute a manipulation pipeline that factors the problem into 1) instance segmentation, 2) 3D keypoint detection, 3) optimization-based robot action planning 4) geometric  ... 
arXiv:1903.06684v2 fatcat:gaghpp3ukjg7xad3u35yplur24

Manipulation-Oriented Object Perception in Clutter through Affordance Coordinate Frames [article]

Xiaotong Chen, Kaizhi Zheng, Zhen Zeng, Shreshtha Basu, James Cooney, Jana Pavlasek, Odest Chadwicke Jenkins
2022 arXiv   pre-print
In our experiments, we demonstrate that ACF outperforms state-of-the-art methods for object detection, as well as category-level pose estimation for object parts.  ...  To achieve this, we aim to provide robust and generalized perception of object affordances and their associated manipulation poses for reliable manipulation.  ...  Compared to 3D keypoints registered on whole objects, our ACF representation decomposes object into object parts based on their functionality, and include explicit orientation for manipulation.  ... 
arXiv:2010.08202v3 fatcat:an3zvvmh3bhira4k23zsejetiu

Generating Task-specific Robotic Grasps [article]

Mark Robson, Mohan Sridharan
2022 arXiv   pre-print
and orientation.  ...  The representation encodes task-specific knowledge for each object class as a relationship between a keypoint skeleton and suitable grasp points that is preserved despite intra-class variations in scale  ...  Third, the work described in this paper built an object representation based on semantic keypoints that provided some robustness to changes in scale and orientation.  ... 
arXiv:2203.10498v1 fatcat:o42v3eukmbdrzgkmjhdkoopbny

Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies [article]

Priya Sundaresan, Jennifer Grannen, Brijen Thananjeyan, Ashwin Balakrishna, Jeffrey Ichnowski, Ellen Novoseller, Minho Hwang, Michael Laskey, Joseph E. Gonzalez, Ken Goldberg
2021 arXiv   pre-print
LOKI uses a learned model of manipulation features to refine a coarse grasp keypoint prediction to a precise, optimized location and orientation, while SPiDERMan uses a learned model to sense task progress  ...  Robot manipulation for untangling 1D deformable structures such as ropes, cables, and wires is challenging due to their infinite dimensional configuration space, complex dynamics, and tendency to self-occlude  ...  The da Vinci Research Kit is supported by the National Science Foundation, via the National Robotics Initiative (NRI), as part of the collaborative research project "Software Framework for Research in  ... 
arXiv:2107.08942v1 fatcat:tw4h72iyg5afneroajhwxf7ula

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo [article]

Thomas Kollar, Michael Laskey, Kevin Stone, Brijen Thananjeyan, Mark Tjersland
2021 arXiv   pre-print
SimNet is evaluated on 2D car detection, unknown object detection, and deformable object keypoint detection and significantly outperforms a baseline that uses a structured light RGB-D sensor.  ...  The underlying model, SimNet, is trained as a single multi-headed neural network using simulated stereo data as input and simulated object segmentation masks, 3D oriented bounding boxes (OBBs), object  ...  Models are trained on the simulated small objects dataset generated with the Basler stereo pair camera model (Section A.1).  ... 
arXiv:2106.16118v1 fatcat:7rofj6dpfnbunpibndqvkhf5be

You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration [article]

Bowen Wen, Wenzhao Lian, Kostas Bekris, Stefan Schaal
2022 arXiv   pre-print
Nevertheless, it often requires expensive real-world data collection and manual specification of semantic keypoints for each object category and task.  ...  Additionally, coarse keypoint predictions and ignoring intermediate action sequences hinder adoption in complex manipulation tasks beyond pick-and-place.  ...  KPAM 2.0 -This is based on the related work [6] that extends KPAM [5] by augmenting semantic keypoints with orientation information for improved expressiveness.  ... 
arXiv:2201.12716v2 fatcat:qbgchamnlvfvhhu6avtulp6yq4

GKNet: grasp keypoint network for grasp candidates detection [article]

Ruinian Xu, Fu-Jen Chu, Patricio A. Vela
2021 arXiv   pre-print
A final filtering strategy based on discrete and continuous orientation prediction removes false correspondences and further improves grasp detection performance.  ...  Contemporary grasp detection approaches employ deep learning to achieve robustness to sensor and object model uncertainty.  ...  Recent work on affordances and keypoints (Xu et al. 2021) indicates that keypoints should work well for recovering SE(3) grasp frames.  ... 
arXiv:2106.08497v3 fatcat:453hdniyanbyborxg66inxxrc4

BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models [article]

Bowen Wen, Kostas Bekris
2021 arXiv   pre-print
Tracking the 6D pose of objects in video sequences is important for robot manipulation.  ...  It leverages the complementary attributes of recent advances in deep learning for segmentation and robust feature extraction, as well as memory-augmented pose graph optimization for spatiotemporal consistency  ...  For instance, via semantic segmentation [40] - [42] or non-semantic methods, such as image (2) a network detects keypoints and their descriptors; (3) keypoints are matched and coarse registration is  ... 
arXiv:2108.00516v1 fatcat:5qxqw2zltveulflnmqgffobfgi

Knowledge-Enabled Robotic Agents for Shelf Replenishment in Cluttered Retail Environments [article]

Jan Winkler, Ferenc Balint-Benczedi, Thiemo Wiedemeyer, Michael Beetz, Narunas Vaskevicius, Christian A. Mueller, Tobias Fromm, Andreas Birk
2016 arXiv   pre-print
and stacked objects with a variety of textures and shapes, (o) knowledge processing methods produce strategies for tidying up supermarket racks, and (o) the necessary manipulation skills in confined spaces  ...  To enable them to act competently, we propose a framework based on three core components: (o) a knowledge-enabled perception system, capable of combining diverse information sources to cope with occlusions  ...  In our experiments we show how the results of the real perception system reflect in the generation of manipulation plans for robotic agents.  ... 
arXiv:1605.04177v1 fatcat:s443qut5tngsrlhwayxt3qjgpy

Vision-based Robotic Grasp Detection From Object Localization, Object Pose Estimation To Grasp Estimation: A Review [article]

Guoguang Du, Kai Wang, Shiguo Lian, Kaiyong Zhao
2020 arXiv   pre-print
This paper presents a comprehensive survey on vision-based robotic grasp detection methods.  ...  In detail, object localization task contains object localization without classification, object detection and object instance segmentation.  ...  and 3D end-to-end feature learning on both geometry and RGB input for 3D object bounding box detection and semantic instance segmentation.  ... 
arXiv:1905.06658v2 fatcat:6u3k2ltwifaanjpp2nkayyj2f4

3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans [article]

Antoni Rosinol, Arjun Gupta, Marcus Abate, Jingnan Shi, Luca Carlone
2020 arXiv   pre-print
We integrate state-of-the-art techniques for object and human detection and pose estimation, and we describe how to robustly infer object, robot, and human nodes in crowded scenes.  ...  Finally, we discuss the implications of our proposal on modern robotics applications. 3D Dynamic Scene Graphs can have a profound impact on planning and decision-making, human-robot interaction, long-term  ...  Second, we match every keypoint on the CAD model with any keypoint on the Kimera model. Clearly, this step produces many incorrect putative matches (outliers).  ... 
arXiv:2002.06289v2 fatcat:le5fcq3yy5d3rjjszewjbrhkti

HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction [article]

Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, Li Yi
2022 arXiv   pre-print
Frame-wise annotations for panoptic segmentation, motion segmentation, 3D hand pose, category-level object pose and hand action have also been provided, together with reconstructed object meshes and scene  ...  With HOI4D, we establish three benchmarking tasks to promote category-level HOI from 4D visual signals including semantic segmentation of 4D dynamic point cloud sequences, category-level object pose tracking  ...  To complete these tasks, participants need to properly plan their actions based on the specific scene configuration.  ... 
arXiv:2203.01577v3 fatcat:kkwisjhrkbgzfp764bt26hd2ra

3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation [article]

Naman Sharma, Hocksoon Lim
2021 arXiv   pre-print
The network is trained to predict the movement of an object based on the correlation features of extracted keypoints across time.  ...  Calculating correlation across keypoints only allows for real-time object detection. We further extend the multi-task objective to include a tracking regression loss.  ...  semantically labeled 3D oriented bounding boxes for all objects in the environment.  ... 
arXiv:2110.02531v1 fatcat:tnh6ef2i6za2hmunvwptlb3g5i

PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data [article]

Zheng Tang, Milind Naphade, Stan Birchfield, Jonathan Tremblay, William Hodge, Ratnesh Kumar, Shuo Wang, Xiaodong Yang
2020 arXiv   pre-print
First, it overcomes viewpoint-dependency by explicitly reasoning about vehicle pose and shape via keypoints, heatmaps and segments from pose estimation.  ...  Second, it jointly classifies semantic vehicle attributes (colors and types) while performing ReID, through multi-task learning with the embedded pose representations.  ...  Synthetic data have been successfully applied to a variety of problems, such as optical flow [17] , car detection [22] , object pose estimation [26, 36] , vision-based robotic manipulation [8, 34]  ... 
arXiv:2005.00673v1 fatcat:jg5gate3lrghbcppbj2jiityue
« Previous Showing results 1 — 15 out of 295 results