The resolution of an image is converted effectively using a simple process which needs a small amount of calculation. Local energy is determined from the differences between pixel values of N pixels lying at diagonal locations in an upper and lower rows. When the local energy is greater than a threshold value, a diagonal line given by two pixels having the smallest difference is employed as an edge direction. An image is interpolated between the two pixels in the edge direction by employing the mean value of the two pixel values of those two pixels. Thereafter, it is determined whether consistency is achieved between the interpolated image and the upper and lower pixels. If consistency is not achieved, linear interpolation is performed as in the case where the energy is lower than the threshold value.