A system and method are provided for contrast-invariant registration of images, the system including a processor, an imaging adapter or a communications adapter for receiving an image data sequence, a user interface adapter for selecting a reference frame from the image sequence or cropping a region of interest (ROI) from the reference frame, a tracking unit for tracking the ROI across the image sequence, and an estimation unit for segmenting the ROI in the reference frame or performing an affine registration for the ROI; and the method including receiving an image sequence, selecting a reference frame from the image sequence, cropping a region of interest (ROI) from the reference frame, tracking the ROI across the image sequence, segmenting the ROI in the reference frame, and performing an affine registration for the ROI.