An image processing apparatus and method, which can appropriately extract an object even when the focus of a camera having an automatic focus adjustment function shifts from the background to the object. To this end, frame images which are sensed by an image sensing unit and are sequential in the time axis direction are input from an input unit. The input frame images are smoothed by a filter unit. A background image generation unit generates an average image of a predetermined number of smoothed frame images as a background image. An image differential unit generates a differential image between the predetermined smoothed frame image and the background image. An object extraction unit extracts an object region where a predetermined object is sensed, on the basis of the differential image.