Various embodiments of the present invention are generally directed to an apparatus and method for non-volatile caching of data in a memory hierarchy of a data storage device. In accordance with some embodiments, a pipeline memory structure is provided to store data for use by a controller. The pipeline has a plurality of hierarchical cache levels each with an associated non-volatile filter cache and a non-volatile victim cache. Data retrieved from each cache level are respectively promoted to the associated non-volatile filter cache. Data replaced in each cache level are respectively demoted to the associated non-volatile victim cache.