Hand Tracking and Gesture Recognition for Interactive Games

Abstract

In this paper, an interactive system having hand-tracking and gesture recognition has been developed in the complex environment. Our developed interactive system can be applied in an interactive game. Our system consists of three modules, such as the foreground detection module, tracking module and recognition module. The foreground detection module is based on the codebook algorithm. The purpose of the foreground detection module is to remove some portions of the image background. It will reduce the interference caused by the background for tracking and recognition. The CAMSHIFT algorithm utilizing the color of the target object for tracking is used in the tracking module. In the recognition module, the Pairwise Geometric Histogram (PGH) of the hand contour is used to describe shape statistics values into a histogram. Finally, the most similar gesture is found by comparing the PGH shape histograms which consist of the PGH obtained from the hand contours and the PGH obtained from the gesture template contours. For PGH, in the experiment, we tested the adaptive scaling and rotation, and the effect of template matching. Our developed system is applied to the interactive game named the finger-guessing game. We use a Webcam to track the movement of hands, determine whether the user selectes or not, and recognize the hand gesture of the user who is playing the game.