A digital video decoding system receives packetized video data representing programs conveyed on a plurality of video channels. The system includes a plurality of buffers for storing encoded video data representing images of video programs conveyed on a corresponding plurality of video channels. An individual buffer, corresponding to an individual video channel, stores sufficient encoded video data to prevent an underflow condition following switching to decode a program conveyed on the individual video channel. A processor initiates switching to decode a program conveyed on a selected one of the plurality of video channels in response to a user channel selection input. A decoder decodes encoded video data received from one of the plurality of buffers corresponding to the program conveyed on the selected video channel as determined by switching initiated by the processor. The decoder also predicts a next channel to be selected by a user based on, (a) predetermined user channel and program preference criteria, (b) predetermined user channel navigation patterns, or (c) user data entry device sensory data.