A modular implant system includes a set of anatomically-designed diaphyseal fitting and filling modular implant components and collars. The diaphyseal component connects with a selected intramedullary stem and with a selected collar component. The collar component connects to another implant component such as a modular articular component, a segmental component or an intercalary component. The diaphyseal component has a tapered porous surface that is received with a tapered bore in the bone diaphysis that is prepared to match the size and shape of the tapered porous surface. The collar component has a porous surface for tissue ingrowth, such as the periosteum, to seal the intramedullary canal. The diaphyseal implant is easy to insert and remove, does not bind before fully seating to prevent stress shielding, and eliminates the long lever arm created when fixation occurs only at the tip of the stem, and should eliminate related stem loosening.