This invention discloses a fast two-dimensional inverse Discrete Cosine Transform (iDCT) that adapts to compressed video source statistics to reduce execution time. iDCT algorithms for sparse blocks eliminate calculations for some zero coefficients and are implemented with quad-word parallel single-instruction-multiple-data (SIMD) multimedia instructions. It is observed that end-of-block marker value histograms vary little within single shots. An adaptive control mechanism is proposed that selects the optimal set of iDCTs to prepare for an entire shot from its first frames (to reduce software overheads and penalties). This introduces no degradation of decoded video quality as compared with a conventional SIMD 8.times.8 iDCT implemented with Intel MMX instructions.