At least one visual significance metric is determined (12) for at least some frames belonging to an original series of frames (11). Key frames are identified (13) as a function, at least in part, of the visual significance metric. Cumulative visual significance values are then determined (14) for at least some of the frames that intervene between each pair of key frames. These cumulative visual significance values are then used to identify (15) frames of additional content interest. Various frames are then selected (16) for use in a visual summary. In one embodiment, all of the key frames and frames of additional content interest are selected for inclusion in the visual summary.