A method (and system) for managing a distributed-memory parallel application running on multiple servers, includes dynamically moving a plurality of executable program components, where each of the plurality of executable program components are running on one of a plurality of virtual machines, using migration of the virtual machines between physical servers. The load balancing is operated based on a workload of each of the virtual machines and servers, where a virtual machine, or a plurality of virtual machines, are transferred to balance the workload between each of the servers.