A marginal distribution feature extraction unit 20 sequentially focuses on each pixel in a reference image and the corresponding pixel in an inspection image, and computes a marginal distribution feature value indicative of the spatial variation in pixel values in the focus pixel neighborhood in both images. Based on this marginal distribution feature value, a tolerance image generation selection unit 26 sets a tolerance range for the reference image or inspection image focus pixel with the less spatial variation. A target image selection unit 30 selects the image comprising pixels from both images for which a tolerance range is not set as a target image. Referencing the set tolerance ranges, a comparison operator 34 compares each pixel in the target image and tolerance image, and outputs a difference Sub representing how far the pixel values of the target image are from the respective tolerance range. A defect determination circuit 38 then outputs data indicating the pixels corresponding to a defect based on a difference map containing these difference Sub values.