A system and method for use with a virtual machine, including an adaptive, automated memory management process that takes decisions regarding which garbage collector technique should be used, based on information extracted from the currently active applications. Reinforcement learning is used to decide under which circumstances to invoke the garbage collecting processing. The learning task is specified by rewards and penalties that indirectly tell the RLS agent what it is supposed to do instead of telling it how to accomplish the task. The decision is based on information about the memory allocation behavior of currently running applications. Embodiments of the system can be applied to the task of intelligent memory management in virtual machines, such as the Java Virtual Machine (JVM).