To improve compression efficiency and decompressed image quality, images are decomposed into separate layers each containing a limited number of image element types (text, line art, or photographic). Each layer can then be compressed separately with a process that is optimal for the type of image element included in the layer. Images are decomposed into foreground, background, and mask layers. The value of a pixel in the mask layer designates whether the spatially corresponding pixel of the recomposed image will be selected from the spatially corresponding pixels of either the foreground or the background layer.