The present invention relates to three-dimensional shape detection. In the present invention, a plurality of types of pattern lights formed of a series of alternate light and dark patterns are projected onto an object in a time series, an image of the object onto which each pattern light is projected is taken, a plurality of luminance images are generated, a code image having certain codes assigned to the pixels is generated in accordance with a result of threshold processing of the plurality of luminance images with respect to a certain threshold; and the three-dimensional shape of the object is calculated. Further, in the invention, a first pixel that is adjacent to a pixel having a code of interest and that has a code different from the code of interest is detected, in a detection position in a direction crossing the pattern light in the code image, a luminance image having a light-dark boundary in a position corresponding to the first pixel is extracted, from the plurality of luminance images, a pixel area that includes a pixel in a certain area adjacent to the first pixel is determined, an approximate expression that expresses a change in luminance in the extracted luminance image in the pixel area is calculated, a position having a certain luminance threshold in the approximate expression and detecting the boundary coordinates of the code of interest in accordance with the result of calculation is calculated, and the three-dimensional shape of the object is calculated in accordance with the boundary coordinates detected by the boundary coordinate detection unit by using the code image.