Transforming optical images of a portion including a normal conductor pattern having a surface roughness, a portion subjected to an inspection, and a reference portion to images of electric charges and picking up these as electric signals by an image pick-up device, rendering the optical image including the normal conductor pattern having the surface roughness to a pixel signal by the image pick-up device, controlling a light volume of the optical image so that the pixel signal is saturated or immediately before the saturation, picking up a pixel signal of the portion to be inspected under this light volume, obtaining a differential signal from a pixel signal picked up from the reference portion, and judging an existence of defect from the differential signal, so as to detect defects such as a hiatus of conductor, a short circuit, and a deposition of an extraneous matter on a wafer, on which the normal conductor pattern having the roughened surface, with a high accuracy in processes of forming films and etching in manufacturing a semiconductor device.