Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection

We present a new method for the 3D model-based tracking of human body parts. To mitigate the difficulties arising due to occlusion among body parts, we employ multiple calibrated cameras in a mutually orthogonal configuration. In addition, we develop criteria for a time varying active selection of a set of cameras to track the motion of a particular human part. In particular, at every frame, each camera tracks a number of parts depending on the visibility of these parts and the observability of their predicted motion from the specific camera. To relate points on the occluding contours of the parts to points on their models we apply concepts from projective geometry. Then, within the physics-based framework we compute the generalized forces applied from the parts' occluding contours to model points of the body parts. These forces update the translational and rotational degrees of freedom of the model, such as to minimize the discrepancy between the sensory data and the estimated model state. We present initial tracking results from a series of experiments involving the recovery of complex 3D motions in the presence of significant occlusion.