A method of manufacturing a thin-film semiconductor device, including forming a crystallized region on a transparent insulating substrate, implanting an impurity into the crystallized region and an amorphous semiconductor layer to form a source diffusion region and a drain diffusion region in the crystallized region, subjecting the resultant structure to heat treatment, thereby not only activating the impurity implanted in the crystallized region and the amorphous semiconductor layer but also restoring crystallinity of only a portion of the amorphous semiconductor layer which is formed on the crystallized region to thereby turn the portion into a polycrystalline semiconductor layer, and subjecting the resultant surface to selective etching to thereby leave only the polycrystalline semiconductor layer and to remove the amorphous semiconductor layer formed on other regions, thereby forming, in a self-aligned manner, a stacked source diffusion layer and a stacked drain diffusion layer.