To improve caching techniques, so as to realize greater hit rates within
available memory, of the present invention utilizes a entropy signature
from the compressed data blocks to supply a bias to pre-fetching
operations. The method of the present invention for caching data involves
detecting a data I/O request, relative to a data object, and then
selecting appropriate I/O to cache, wherein said selecting can occur with
or without user input, or with or without application or operating system
preknowledge. Such selecting may occur dynamically or manually. The
method further involves estimating an entropy of a first data block to be
cached in response to the data I/O request; selecting a compressor using
a value of the entropy of the data block from the estimating step,
wherein each compressor corresponds to one of a plurality of ranges of
entropy values relative to an entropy watermark; and storing the data
block in a cache in compressed form from the selected compressor, or in
uncompressed form if the value of the entropy of the data block from the
estimating step falls in a first range of entropy values relative to the
entropy watermark. The method can also include the step of prefetching a
data block using gap prediction with an applied entropy bias, wherein the
data block is the same as the first data block to be cached or is a
separate second data block. The method can also involve the following
additional steps: adaptively adjusting the plurality of ranges of entropy
values; scheduling a flush of the data block from the cache; and
suppressing operating system flushes in conjunction with the foregoing
scheduling step.