A system and process for determining the similarity in the shape of objects is presented that generates a novel shape representation called a directional histogram model. This shape representative captures the shape variations of an object with viewing direction, using thickness histograms. The resulting directional histogram model is substantially invariant to scaling and translation. A matrix descriptor can also be derived by applying the spherical harmonic transform to the directional histogram model. The resulting matrix descriptor is substantially invariant to not only scaling and translation, but rotation as well. The matrix descriptor is also robust with respect to local modification or noise, and able to readily distinguish objects with different global shapes. The typical applications of the directional histogram model and matrix descriptor include recognizing 3D solid shapes, measuring the similarity between different objects and shape similarity based object retrieval.