A method determines a shape of an object in from an image. A visual hull larger than the object is constructed such that the object can be entirely contained within the visual hull. An image is acquired of the object by a camera. The visual hull is projected onto the image according to a position and pose of the camera.Seed pixels in the image are selected. The seed pixels are located on the projected visual hull, and a region is grown around each seed pixel. Each region includes neighboring pixels resembling the seed pixel. Then, the visual hull is reduced in size according to the grown regions to determine a two-dimensional shape of the object. When multiple images are used from different points of view, the three-dimensional shape can be determined.