The present invention relates to a magnetoresistive effect element, which has a large MR ratio, excellent thermostability and a small switching magnetic field even if its size is decreased, and a magnetic memory using the magnetoresistive effect element. The magnetoresistive effect element includes: a storage layer formed by stacking a plurality of ferromagnetic layers via non-magnetic layers; a magnetic film having at least one ferromagnetic layer; and a tunnel barrier layer provided between the storage layer and the magnetic film. Each of the ferromagnetic layers of the storage layer is formed of an Ni--Fe--Co ternary alloy. A peak-to-peak maximum surface roughness on each of an interface between the storage layer and the tunnel barrier layer and an interface between the magnetic film and the tunnel barrier layer is 0.4 nm or less.