A system and methodology for removing JPEG-compression artifacts from color images, wherein a first-stage examination of the image takes place to determine, in accordance with several approaches, whether artifact removal is appropriate for the image. Following such examination and characterization, when an image is determined to be a candidate for artifact removal, pixel blocks in the image are examined so that they can be characterized as uniform, busy or transitional, and from these characterizations, appropriate filters, or no filter at all, are employed to prepare such pixels for insertion into a final artifact-removed output color image.