A dynamic allocation of available ASV buffer memory space is performed on each pack in a DVD-A bitstream one pack at a time. Concurrently, an ASV buffer table is updated for each type of data pack currently being processed. The ASV buffer table includes pointers corresponding to the various fields that form a particular ASV frame. In this way, only that memory that is required to store a particular ASV frame is used thereby allowing the ASV buffer memory to be configured on the fly in such a manner as to efficiently store the required ASV frame data. When a particular ASV frame is to be displayed, or otherwise processed, the ASV buffer table is accessed, and the particular pointers for a specific ASV frame are looked up and used to access the desired ASV frame.