When first and second images are input, a partial image feature calculating unit calculates feature values of partial images of the two images. A maximum matching score position searching unit searches for a position of the second image that attains to the highest matching score with each of the partial images of the first image. A movement-vector-based similarity score calculating unit calculates similarity between the first and second images, using information related to that partial image whose movement vector has direction and length within a prescribed range, which movement vector representing positional relation between a reference position for measuring, for each of the partial images, the position of the partial image in the first image and the position of the maximum matching score corresponding to the partial image searched out by the maximum matching score position searching unit. The images as the object of collation may belong to the same category classified based on the feature values.