The color of a surface is determined by tracking the position of the surface using grayscale image sensing and then using captured grayscale image information, position information, and grayscale-to-color conversion information to determine color. In one embodiment, the technique requires establishing grayscale-to-color conversion information and a relationship between position and the grayscale-to-color conversion information. A calibration pattern is printed on a piece of paper and the surface of the paper is illuminated. Grayscale image information and position information related to the surface is generated in response to the illumination. Color-specific portions of the grayscale-to-color conversion information that are to be used in color determination are selected based on the position information. Grayscale image information related to the calibration pattern is then compared to the selected color-specific portions of the grayscale-to-color conversion information to determine the color of the calibration pattern.