|
Model-based 3D pose and shape estimation methods reconstruct a full 3D mesh for the human body by estimating several
parameters. However, learning the abstract parameters is a highly non-linear process and suffers from image-model
misalignment, leading to mediocre model performance. In contrast, 3D keypoint estimation methods combine deep CNN
network with the volumetric representation to achieve pixel-level localization accuracy but may predict unrealistic body
structure. In this paper, we address the above issues by bridging the gap between body mesh estimation and 3D keypoint
estimation. We propose a novel hybrid inverse kinematics solution (HybrIK). HybrIK directly transforms accurate 3D
joints to relative body-part rotations for 3D body mesh reconstruction, via the twist-and-swing decomposition. The swing
rotation is analytically solved with 3D joints, and the twist rotation is derived from the visual cues through the
neural network. We show that HybrIK preserves both the accuracy of 3D pose and the realistic body structure of the
parametric human model, leading to a pixel-aligned 3D body mesh and a more accurate 3D pose than the pure 3D keypoint
estimation methods. Without bells and whistles, the proposed method surpasses the state-of-the-art methods by a large
margin on various 3D human pose and shape benchmarks. As an illustrative example, HybrIK outperforms all the previous
methods by 13.2 mm MPJPE and 21.9 mm PVE on 3DPW dataset.
|
