A highly reliable defect automatic classifying method and apparatus capable of flexibly coping with a request for classifying a defect given by each user without having to collect lots of teach data items. A classifying class arrangement is defined by a user by combining classes supplied by the system itself or classes defined by the user and, further, a priori knowledge on the defect class is given by the user as a restriction so as to carry out restricted learning.