|
Recovering whole-body mesh by inferring the abstract pose and shape parameters from visual content can obtain 3D bodies
with realistic structures. However, the inferring process is highly non-linear and suffers from image-mesh misalignment,
resulting in inaccurate reconstruction. In contrast, 3D keypoint estimation methods utilize the volumetric
representation to achieve pixel-level accuracy but may predict unrealistic body structures. To address these issues,
this paper presents a novel hybrid inverse kinematics solution, HybrIK, that integrates the merits of 3D keypoint
estimation and body mesh recovery in a unified framework. HybrIK directly transforms accurate 3D joints to body-part
rotations via twist-and-swing decomposition. The swing rotations are analytically solved with 3D joints, while the twist
rotations are derived from visual cues through neural networks. To capture comprehensive whole-body details, we further
develop a holistic framework, HybrIK-X, which enhances HybrIK with articulated hands and an expressive face. HybrIK-X is
fast and accurate by solving the whole-body pose with a one-stage model. Experiments demonstrate that HybrIK and
HybrIK-X preserve both the accuracy of 3D joints and the realistic structure of the parametric human model, leading to
pixel-aligned whole-body mesh recovery. The proposed method significantly surpasses the state-of-the-art methods on
various benchmarks for body-only, hand-only, and whole-body scenarios.
|
