In certain aspects, the invention features a system and method for caching results, including receiving a job for computation by a distributed computing system having one or more node computing devices in communication with a cache, processing, on one of the node computing devices, the job to create an intermediate result for storage in the cache, wherein the intermediate result includes data wherein a time required to obtain the data by computation or retrieval from a data storage external to the distributed computer system is at least marginally greater than that of retrieving the intermediate result from the cache. In accordance with such aspects, the system and method further includes storing the intermediate result in the cache, and accessing the cache by presenting a lookup function to the cache, wherein the lookup function includes a key and a compute function configured to produce the intermediate result.