A focus detection apparatus capable of accurate focus detection is disclosed. The apparatus includes a first image-forming lens unit including a first pair of lens portions forming a first pair of images on a first pair of light-receiving element arrays with luminous fluxes from an image-pickup optical system after the luminous fluxes pass through a first pair of apertures making a pair in a first direction, and a second image-forming lens unit including a second pair of lens portions forming a second pair of images on a second pair of light-receiving element arrays with luminous fluxes from the image-pickup optical system after the luminous fluxes pass through a second pair of apertures making a pair in a second direction. A midpoint between the second pair of lens portions is located between the first pair of lens portions and displaced from a midpoint between the first pair of lens portions.