On a silicon layer of an SOI wafer is defined a semiconductor device-forming region to form semiconductor devices thereon and an insulating region to electrically insulate the semiconductor device-forming region. Then, a mask layer is formed of nitride by means of photolithography so as to cover the semiconductor device-forming region. Then, an impurities-removing layer is formed by means of well known technique so as to cover the mask layer and embed the gaps between the adjacent masks of the mask layer. The impurities of the silicon layer of the SOI wafer are absorbed and removed by the distorted layer, the grain boundaries and the lattice defects of the impurities-removing layer.