Real-Time 3D Head Motion Tracking Using a Camera

Abstract

In this paper, a real-time 3D head motion tracking technique is presented. Two issues are addressed: feature tracking, and 3D head motion estimation. In feature tracking, three facial features, two eyes and one nostril, are tracked in a live video sequence under regular office lighting condition. In 3D head motion estimation, knowledge about human head motion is involved in developing a cost function that is then minimized iteratively to find a set of 3D motion parameters. The involvement of head motion knowledge is important, since most optimization techniques are prone to be trapped into incorrect local optimal solutions, if only three points correspondence is given. The recovered 3D motion includes rotations about and translations along three axes. A model-based visual communication system that applies the proposed real-time 3D head model moves according to a real person's head movement in 3D space. The system performance is above 25 frames/sec on a regular PC with a Pentium-II 400 MHz CPU, with 3D head motion tracking and texture-mapped head model rendering involved.