A method of scaling up an image. The method converts a source image with M*N pixels into a target image with KM*HN pixels and includes the steps of: choosing a pixel from the source image as a processing pixel; analyzing the character of the processing pixel according to values of a plurality of pixels neighboring the processing pixel, and choosing a tilt pattern as a characteristic pattern; calculating a pixel compensation matrix according to the characteristic pattern, the pixel value of the processing pixel, and the values of the pixels neighboring the processing pixel; calculating K*H target pixel values as the target image according to the pixel value of the processing pixel and its corresponding pixel compensation matrix; and repeating the above-mentioned steps until all the pixels of the source image are processed.