A method and system for detecting faces at different views within images that allocates the computational effort based on a prior probability associated with a view is provided. A face detection system determines whether an image contains a face using detectors that are adapted to detect faces at various views and a filter that filters out windows of the image that are provided to a detector based on a prior probability associated with the view of the detector. Each view has an associated prior probability that a face from a collection of real-life home photographs will be at that view. The face detection system allocates increasing computational effort to a detector as the prior probability of its view increases.