Provided are a method, system, and article of manufacture for adjusting parameters used to prefetch data from storage into cache. Data units are added from a storage to a cache, wherein requested data from the storage is returned from the cache. A degree of prefetch is processed indicating a number of data units to prefetch into the cache. A trigger distance is processed indicating a prefetched trigger data unit in the cache. The number of data units indicated by the degree of prefetch is prefetched in response to processing the trigger data unit. The degree of prefetch and the trigger distance are adjusted based on a rate at which data units are accessed from the cache.