Without using an interferometer, small displacement and/or three-dimensional shape of an object is detected in a noncontact way with high accuracy using pseudo-phase information calculated from e.g., a speckle pattern having a spatially random structure. A speckle image of the test object of the before displacement is obtained, and a spatial frequency spectrum is calculated by executing an N-dimensional Fourier transform for this. The complex analytic signal is obtained by setting the amplitude of frequency spectrum in the half plane including zero frequency in this amplitude distribution to zero, and executing the frequency spectrum amplitude in the half plane of the remainder in the inverse Fourier transform. And then, the amplitude value of this complex analytic signal is replaced with the constant value, a part of the obtained analytic signal domain is clipped, the phase information is calculated by the phase-only correlation function, and the cross-correlation peak in N-dimension is obtained. The displacement magnitude can be obtained by executing the above-mentioned method to the after displacement of the test object, and obtaining the difference of the cross-correlation peak before and after the displacement.