An object is to reduce the number of high temperature (equal to or greater than 600.degree. C.) heat treatment process steps and achieve lower temperature (equal to or less than 600.degree. C.) processes, and to simplify the process steps and increase throughput in a method of manufacturing a semiconductor device. With the present invention, a barrier layer, a second semiconductor film, and a third semiconductor film containing a noble (rare) gas element are formed on a first semiconductor film having a crystalline structure. Gettering is performed and a metallic element contained in the first semiconductor film passes through the barrier layer and the second semiconductor film by a heat treatment process, and moves to the third semiconductor film. The second semiconductor film and the third semiconductor film are then removed, with the barrier layer used as an etching stopper.