The present invention targets a spectral optical element that outputs a light inputted at a predetermined incident angle to a predetermined direction depending on its wavelength. A spectral optical element according to the present invention includes a transmission diffraction optical element, and an optical phase control element that is fixed and arranged to be opposed to a diffracted light output surface of the transmission diffraction optical element so that the diffracted light is inputted therein. Then, this optical phase control element is constructed so that an optical length with respect to a light passing through the optical phase control element is changed in accordance with change of a diffraction angle of the diffracted light depending on the wavelength.