Systems and methods to estimate head motion between two images of a face are described. In one aspect, locations of a plurality of distinct facial features in the two images are identified. The locations correspond to a number of unknowns that are determined upon estimation of head motion. The number of unknowns are determined by a number of equations. The identified locations are converted into a set of physical face parameters based on the symmetry of the distinct facial features. The set of physical face parameters reduce the number of unknowns as compared to the number of equations used to determine the unknowns. An inequality constraint is added to a particular face parameter of the physical face parameters, such that the particular face parameter is constrained within a predetermined minimum and maximum value. The inequality constraint is converted to an equality constraint using a penalty function. Head motion is estimated from identified points in the two images. The identified points are based on the set of physical face parameters.