An image processing apparatus for processing image data, obtained from an image sensing unit in which plural sensor chips each having plural pixels are arrayed with an interval of a multiple of a pixel width therebetween, and interpolating image data for the interval. If an abnormal pixel exists, image data of the abnormal pixel is replaced with data in accordance with image data of a pixel adjacent to the abnormal pixel, and thereafter, interpolation is performed for the interval by using image data from a sensor chip adjacent to said interval.