There are performed three types of object shooting, i.e., a projection pattern shooting using a pattern including both characterization of a stripe based on a color and characterization based on a gradation of the same component, a flash shooting without a pattern and a non-flash shooting without a pattern. A two-dimensional image information generation portion corrects the influence of a surface reflectivity of an object or external light from the three types of images, and estimates a pattern structure using the characteristic based on a color with respect to an object having a white-based color or a low-saturation color and based on a gradation with respect to an object having a high-saturation color. A depth calculation portion specifies the correspondence relationship between the estimated pattern structure and information of the projected pattern, and calculates a depth of each part of the object based on a result of specification.