A method and apparatus for managing blocks of data in a data processing system, the data processing system including a host computer and data processing hardware, the host computer having a main memory that stores blocks of data to be processed by the data processing hardware, the data processing hardware including a local memory that locally stores a subset of the blocks of data to be processed by the data processing hardware. When a portion of one of the blocks of data is to be processed by the data processing hardware, a determination is made as to whether the block of data is in the local memory. When the block of data is in the local memory, the portion of the block of data to be processed is read from the local memory. When the block of data is not in the local memory, it is downloaded from the host computer main memory to the data processing hardware. The data processing hardware may generate an interrupt to the host computer with a request to download data.