An interpolated image obtained by enlarging and interpolating an image (IN) is subjected to predetermined vector transformation to calculate an image feature vector. Also, a characteristic of a subject is obtained to calculate a physical characteristic parameter of the interpolated image. The physical characteristic parameter is used to transform the image feature vector to sharpen the interpolated image. The transformed image feature vector is subjected to transformation inverse to the predetermined vector transformation to generate an enlarged image (ELI).