A method and system for selecting image elements, such as image pixels or vectors, of a digital image provide a set of lexical classifiers, such as a set of color names, which characterize a visual attribute of the image elements such as color. A lexical classifier is assigned to each image element, e.g. by a transform between the image element color values under the original numerical color encoding model of the image and the set of lexical classifiers. A reference lexical classifier is selected, e.g. by selecting an image element and determining its associated lexical classifier, and the image elements are then selected if their lexical classifier corresponds to the reference lexical classifier. The selected image elements may then be acted on as a group.