A method and apparatus to process erroneous bi-directionally predicted multimedia data is described. The method includes determining temporal prediction data for the erroneous multimedia data, and estimating the erroneous multimedia data based on the determined temporal prediction data. The temporal prediction data includes data that was used to predict a forward reference section and/or a backward reference section. In some aspects, the temporal prediction data comprises motion vector data of the reference frames which is scaled to correspond to a frame containing the erroneous data.