A method of encoding a local time base embedded in the compressed data is disclosed. The local time base is encoded in two parts. The first part has a modulo time base that indicates the specific interval in the reference time base and the second part has a time base increment relative to the reference time. Two forms of time base increment is used to allow for the possibility of different encoding order and displaying order. A mechanism for the synchronization of multiple compressed streams with local time base is also described. A time base offset mechanism is also used to allow finer granularity synchronization of the multiple compressed streams.