A method of correcting errors in motion vectors due to camera panning includes identifying an error block that contains an error motion vector out of a plurality of blocks in a current video frame, each block in the current video frame containing a corresponding motion vector. The method also includes calculating an average motion vector value of the motion vectors for all blocks in the current video frame that are not error blocks, identifying a group of effective motion vectors in the current video frame having corresponding motion vector values within a threshold value of the average motion vector value, calculating a panning motion vector value to be equal to an average value of the effective motion vectors, and correcting the error block by setting the motion vector of the error block to be equal to the panning motion vector value.