The present invention relates to a system for searching and browsing multimedia, and more particularly, to a video skimming method and apparatus which is capable of summarizing the full content of a subject video data within a short time and rapidly moving to a desired section by skimming the content of the video based on scenes and shots formed by shot clustering and shot segmentation. The video skimming apparatus selects scenes to be reproduced and scenes to be skipped based on content-based video skimming, and then continuously reproducing a pertinent section in a shot of the scene or partially reproducing the same by utilizing a skipping technique.