A system and method for monitoring disease progression or response to therapy using multi-modal visualization are provided. The method comprises: selecting a first image dataset of a first timepoint; loading the first image dataset of the first timepoint; selecting a second image dataset of a second timepoint; loading the second image dataset of the second timepoint; registering the first image dataset of the first timepoint and the second image dataset of the second timepoint; and displaying the first image dataset of the first timepoint and the second image dataset of the second timepoint.