TY - GEN
T1 - 3D HAND JOINT AND GRASPING ESTIMATION FOR TELEOPERATION SYSTEM
AU - Qi, Liyuan
AU - Popoola, Olaoluwa
AU - Wang, Jingyan
AU - Imran, Muhammad
AU - Ahmad, Wasim
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Gesture-based teleoperation is a complex and essential task that enables remote object manipulation. Recent advancements in 3D human hand pose estimation, driven by affordable depth cameras, have proven its aptitude for this task. However, while previous vision-based approaches focus on mapping hand posture to end-effectors, they overlook the interaction between the robot and the object. This leaves a challenge in interpreting these hand joint estimates into practical robotic behaviour. In this paper, we propose a method that leverages the geometric information of the human hand to enable robots to perform human-like grasping and manipulation. Our approach incorporates a pointcloud-based hand joint regressor and the grasping direction analysis (GDA) to control the robot. The joint-wise regressor showed an improved mean joint error of 7.8mm on the MSRA dataset compared to the 8.5mm baseline. We demonstrate that the GDA-based teleoperation can successfully perform real-time robotic manipulator controlling and grasping for various tasks.
AB - Gesture-based teleoperation is a complex and essential task that enables remote object manipulation. Recent advancements in 3D human hand pose estimation, driven by affordable depth cameras, have proven its aptitude for this task. However, while previous vision-based approaches focus on mapping hand posture to end-effectors, they overlook the interaction between the robot and the object. This leaves a challenge in interpreting these hand joint estimates into practical robotic behaviour. In this paper, we propose a method that leverages the geometric information of the human hand to enable robots to perform human-like grasping and manipulation. Our approach incorporates a pointcloud-based hand joint regressor and the grasping direction analysis (GDA) to control the robot. The joint-wise regressor showed an improved mean joint error of 7.8mm on the MSRA dataset compared to the 8.5mm baseline. We demonstrate that the GDA-based teleoperation can successfully perform real-time robotic manipulator controlling and grasping for various tasks.
KW - hand gesture recognition
KW - hand joints estimation
KW - human-machine interface
KW - point cloud analysis
KW - teleoperation
UR - https://www.scopus.com/pages/publications/85195408279
U2 - 10.1109/ICASSP48485.2024.10448299
DO - 10.1109/ICASSP48485.2024.10448299
M3 - Conference contribution
AN - SCOPUS:85195408279
T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
SP - 4365
EP - 4369
BT - 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024
Y2 - 14 April 2024 through 19 April 2024
ER -