A method of extracting key frames from a video sequence, which video sequence includes MPEG compressed video data having block motion vectors. The method initially partially decompresses (202) the MPEG compressed video data to obtain block motion vectors and converts (204) the block motion vectors to forward block motion vectors. The method then generates (206) global motion signals and generates (306) dominant global direction clusters. The method then selects (402,404,406) potential key frames of the video sequence using the generated dominant global direction clusters. The method lastly decompresses (408) the selected key frames to obtain the extracted key frames.