In an image processing apparatus, color image data is binarized, and a symbol area in the color image data is detected. There are obtained, from N colors that form a symbol in the symbol area, M colors that are equal to or smaller than N in number. A symbol cutting process is performed on the symbol area, and one of the M colors is allocated for each symbol cut unit that is obtained by the symbol cutting.