One embodiment of the present invention provides a memory controller that contains a distributed cache that stores cache lines for pending memory operations. This memory controller includes an input that receives memory operations that are directed to an address in memory. It also includes a central scheduling unit and multiple agents that operate under control of the central scheduling unit. Upon receiving a current address, a given agent compares the current address with a cache line stored within the given agent. All of the agents compare the current address with their respective cache line in parallel. If the addresses match, the agent reports the result to the rest of the agents in the memory controller, and accesses data within the matching cache line stored within the agent to accomplish the memory operation.