A method for non-rigid registration and fusion of images with physiological modeled organ motions resulting from respiratory motion and cardiac motion that are mathematically modeled with physiological constraints. A method of combining images comprises the steps of obtaining a first image dataset (24) of a region of interest of a subject and obtaining a second image dataset (34) of the region of interest of the subject. Next, a general model of physiological motion for the region of interest is provided (142). The general model of physiological motion is adapted with data derived from the first image data set (140) to provide a subject specific physiological model (154). The subject specific physiological model is applied (172) to the second image dataset (150) to provide a combined image (122).