The position of an eye is detectable with high precision from a face image of a person taken under near infrared illumination or the like. After pre-processing, the face image is subjected to brightness correction to increase the contrast between the sclera portion and iris portion of the eye. Brightness gradient vectors are calculated for the brightness-corrected image, and matching is performed between a brightness gradient image generated using the calculated brightness gradient vectors and an eye template. Further, matching with a pupil template is performed to correct the eye center position. Final positions of both eyes are then determined.