When an STI element isolation structure is formed, it is formed in such a manner that its upper portion protrudes further than the surface of a substrate than by a normal STI method, and a dummy electrode pattern is formed in a gate electrode forming portion. After a source/drain is formed in alignment with a gap portion, a conductive layer formed by filling the gap portion with W is formed, the dummy electrode pattern is removed, and a gate insulating film and a gate electrode are formed.