The invention provides a panel fixing plate as an optical modulation device retainer having a substantially c-shaped fixing section body viewed from the top, an arm section formed at a tip side of the fixing section body, a pin protruding from the arm section, and a spring member biasing an emission-side polarization plate as an optical conversion element with respect to the fixing section body. Liquid crystal panels, each as light modulators, are securely fixed to a panel fixing plate with the pin inserted into each corresponding hole. The emission-side polarization plate can be inserted into the substantially c-shaped fixing section body when viewed from the top so as to be biased and fixed to the fixing section body by the spring member. An optical modulation device retainer, an optical device, and a projector of this invention, light modulators and/or optical conversion elements can be exchanged with ease, and the reworkability can be improved.