A focus detecting system for determining the focus state of an imaging lens, such as a camera lens, is disclosed. Object light not forming an image for image production is deflected along an optical axis through a relay lens to a focus state detection unit that includes image detecting elements with light receiving surfaces that are equidistant in front of and behind a position conjugate with a light receiving surface for image production. Either the relay lens or the focus state detection unit is moveable along the optical axis to vary how far the light receiving surfaces are out of focus. A diaphragm opening of the imaging lens determines if and by how much the relay lens and focus state detection unit are relatively moved. Evaluation values of the out of focus amount are compared in order to control focusing of the imaging lens.