An osmotic delivery system has a membrane plug retention mechanism which can also be used to control the delivery rate of a beneficial agent from the osmotic delivery system. The osmotic delivery device includes an implant capsule containing a beneficial agent and an osmotic agent. Holes are formed along a side wall of the implant capsule at an open end of the capsule. When the membrane plug is inserted into the open end of the capsule, the membrane material swells into the holes in the capsule side wall creating a large frictional force which prevents expulsion of the membrane plug. A beneficial agent delivery rate of the osmotic delivery system is controllable by varying the size and number of the holes to change the amount of exposed surface area of the membrane plug. An increase in the surface area of the membrane plug exposed to the exterior environment causes a corresponding increase in the liquid permeation rate of the membrane and thus, increases the beneficial agent delivery rate.