Memory allocation to multiple computing units is disclosed. A static offset for each computing unit is determined, and a portion of memory is allocated for each computing unit, and remapped into a contiguous logical region that is addressable by a pointer plus the static offset. The portion of the memory is dynamically passed out to each computing unit as the computing units need memory. Upon the initial contiguous memory being completely passed out to the computing units, a number of physically non-contiguous sections of memory are mapped into another logically contiguous section of memory. A portion of this logically contiguous section of memory is allocated for each computing unit, and is addressable by a pointer plus the static offset that was previously determined. The portion of the logically contiguous section of memory can be dynamically passed out to each computing unit as the computing units need memory.