Markerless Upper Limb Motion Tracking: A Comparative Evaluation of Multi-View Approaches

This work presents a comparative evaluation of markerless upper-limb motion tracking methods, namely, Medi-aPipe (monocular), Fusion (multi-view fusion of MediaPipe out-puts), and Triangulation (3D reconstruction via calibrated multi-view geometry), against a marker-based ground truth acquired with OptiTrack. The objective is to assess the feasibility and accuracy of these approaches in reconstructing 3D joint positions during human motion imitation tasks. Data were collected using RGB-D cameras and compared with OptiTrack data after time-based synchronization. This study uses only RGB data. Four representative gestures, Drinking, Eating with a utensil, Closing a zipper, and Touching the nose, were analyzed. The evaluation focuses on the positional error of the shoulder, elbow, and wrist joints, using RMSE as the primary metric. Preliminary results indicate that triangulation achieves the highest spatial accuracy when MediaPipe detections are consistent across views, but its performance degrades rapidly under occlusions or missing key-points. In contrast, MediaPipe and multi-view fusion offer greater robustness in such conditions, maintaining functional accuracy even when visibility is compromised. These findings inform the design of adaptable, markerless motion capture pipelines suitable for different human-robot interaction scenarios.