A gate electrode is formed on a semiconductor substrate with a gate insulating
film interposed therebetween, and a sidewall spacer is then formed at the lateral
sides of the gate electrode on the semiconductor substrate. Epitaxial growth is
conducted at a lower growth rate to form, at both lateral sides of the sidewall
spacer on the semiconductor substrate, first semiconductor layers made of first
single-crystal silicon films superior in crystallinity. Then, epitaxial growth
is conducted at a higher growth rate to form, on the first semiconductor layers,
second semiconductor layers made of single-crystal films or polycrystalline films,
which are inferior in crystallinity, or amorphous films. The upper areas of the
first semiconductor layers and the whole areas of the second semiconductor layers
are doped with impurity, thus forming impurity diffusion layers respectively serving
as a source and a drain.