Optical isolator comprises a ferrule having a through-hole in which an optical fiber is inserted and a groove formed in the ferrule with the optical fiber inserted in the through-hole; and a composite assembly consisting of an optical isolator element composed of a Faraday rotator and a pair of polarizers fixed on opposite surfaces of the Faraday rotator in such manner that the angle between the directions of polarization of the polarizers is at about 45 and a structure reinforcing member having a thickness of filling the groove and surrounding the optical element, said composite assembly being inserted in the groove as a unit. The reinforcing member may be made of the same material as the outer support or formed by a magnet for applying a magnetic field to the Faraday rotator. The isolator element, the magnet and the outer support may have respective reference sides or planes so that the plane of polarization of the incident side of the isolator element is easily determined with respect an outer base or support. Also, the groove portion of the support is mechanically strengthened by the magnet surrounding the optical element.