A prospective abnormal shadow detecting system detects a prospective abnormal shadow in an image on the basis of image data representing the image and outputs information on the prospective abnormal shadow detected by the prospective abnormal shadow detecting means. Whether the detected prospective abnormal shadow is malignant or benignant is judged, and the information on the prospective abnormal shadow is output in such a manner that whether the prospective abnormal shadow is malignant or benignant can be distinguished.