A method and apparatus for downscaling video images to a lower resolution (e.g. from HDTV to SDTV) is presented. The method comprising the steps of frequency domain anti-aliasing filtering and downscaling the first video signal in a first direction corresponding with a line direction in the first video signal to obtain a downscaled video signal, and spatial domain downscaling the downscaled video signal in a second direction perpendicular to the first direction to obtain the second video signal. The method and apparatus of the invention are suitable for efficient and high quality decoding both progressive and any-type encoded interlaced signals.