A swelling packer system uses modules that can be joined together and mounted over a tubular using a vertical split that can be drawn closed with a tapered pin in overlapping loops. The pins are circumferentially spaced apart as between adjacent modules. End rings can protect the modules for run in and act as extrusion barriers during and after the swelling is complete. The module ends can be overlapped in an interlocking fashion which allows multiple elements to be joined together to make a packer assembly as long as desired with any combination of swelling elements in a single packer assembly. Optionally, interior grooves in the swelling material can hold split ring seals or o-ring type seals that are slipped over a tubular end before a module is clamped on. The sealing elements can be triggered with water or hydrocarbons or with other materials already in the wellbore or introduced to it or other surface or locally actuated triggers.