A method detects an object, such a face, in an image. The image is first partitioned into patches of various sizes using either an integral image or a Gaussian pyramid. Features in each patch are evaluated to determine a cumulative score. The evaluating is repeated while the cumulative score is within a range of an acceptance threshold and a rejection threshold, and otherwise the image is rejected when the accumulated score is less than the rejection threshold and accepted as including the object when the cumulative score is greater than the acceptance threshold.