A video recording and playback device (10) incorporates a processor (114) for processing the video signal to detect the presence of commercial messages. Video and audio event detectors (102, 104) detect the presence of events in the signal as it is recorded. The timing relationship of the detected events is analyzed to classify the video segments between events as program material or as commercial messages. After a program is recorded, control marks are generated (132) to indicate the beginning and end of commercial groups so that they will be skipped during subsequent replay of the recorded program.