In one embodiment, a scene change detection component of a video encoder uses motion vectors to determine whether a scene change occurs in a video sequence. The scene change detection component uses field motion vectors determined by a motion estimator and compares the field motion vectors to a threshold to determine whether a scene change occurs. In another embodiment, if a scene change occurs, the video encoder can begin a new Group of Pictures (GOP).