An index detection unit (2030) detects indices allocated or set on an object from a sensed image. An evaluation amount calculation unit (2060) calculates evaluation amounts of the indices using two-dimensional geometric features of the indices on the image and/or three-dimensional geometric features that represent relationships between an image sensing device (2010) and the indices on a three-dimensional space. A reliability calculation unit (2070) calculates reliabilities of the indices according to the calculated evaluation amounts of the indices. A position and orientation calculation unit (2080) calculates the position and orientation of the object or the image sensing device (2010) using at least the calculated reliabilities of the indices and information associated with the image coordinates of the detected indices.