A semiconductor device includes a semiconductor region having a source region, a drain region, and a channel region provided between the source region and the drain region, a first tunnel insulation film formed on the channel region, a barrier layer formed on the first tunnel insulation film and having an energy barrier, a second tunnel insulation film formed on the barrier layer, a charge storage portion formed on the second tunnel insulation film and comprising an insulation film expressed by Si.sub.Y(SiO.sub.2).sub.X(Si.sub.3N.sub.4).sub.1-XM.sub.Z (where, M denotes an element other than Si, O, and N, and 0.ltoreq.X.ltoreq.1, Y>0, and Z.gtoreq.0), and a control electrode formed on the charge storage portion and controlling a height of the energy barrier.