A focus detection apparatus is disclosed, which realizes both of vertical line detection and horizontal line detection over a wide area of an image-taking range in focus detection of a phase difference detection method. The focus detection apparatus includes an optical system, which divides a luminous flux from an image-taking optical system to form a first pair of optical images having an interval between them in a first direction and a second pair of optical images having an interval between them in a second direction, and a photoelectrical conversion device. The optical system includes an optical member, which provides a first optical action widening the interval between the first pair of the optical images in the first direction. The first optical action is different from a second optical action for the second pair of the optical images in the second direction.