An image processing apparatus displays a realistic image depending on a user's viewpoint. A sensor control unit 41 detects a user's viewpoint and a point of interest based on signals from various sensors, and outputs the detected viewpoint and point to a depth-of-field adjusting unit. The depth-of-field adjusting unit reads, from the image database, a depth of field set beforehand (or designated each time) and image data of images captured from different positions according to user's viewpoints, and outputs the read depth of field and data to an image combining unit. The image combining unit combines a plurality of image data items output from the depth-of-field adjusting unit and displays a combined image on a display. The present invention is applicable to a television apparatus.