A low spatial frequency video sequence is enhanced to provide a higher spatial frequency video sequence using a super-resolution process. Patches of higher frequency image data are inferred from the images of the lower frequency video sequence and a dictionary containing a training set of higher resolution image data. An inference module selects result patches from the training set to preserve spatial consistency within each image frame and to preserve temporal consistency, between image frames of the resulting video sequence. Temporal consistency can be preserved for static portions and/or moving portions of each frame.