A method and system for improving the accuracy and timeliness of video metadata by incorporating information related to the motion of the camera as derived from the video imagery itself. Frame-to-frame correspondences are used to accurately estimate changes in camera pose. While the method and system do not require geo-registration, geo-registration results, if available, may be considered in processing the video images and generating improved camera pose estimates.