The pattern defect inspection apparatus is operable to detect defects by comparing a detection image, which is obtained through scanning by an image sensor those patterns that have the identical shape and are continuously disposed on the object under tested at equal intervals in row and column directions, with a reference image obtained by scanning neighboring identical shape patterns in the row and column directions. This apparatus has a unit for generating an average reference image by statistical computation processing from the images of identical shape patterns lying next to the detection image including at least eight nearest chips on the up-and-down and right-and-left sides and at diagonal positions with the detection image being intermediately situated. The apparatus also includes a unit that detects a defect by comparing the detection image to the average reference image thus generated.