This invention relates to an endoscope image sensing apparatus which can switch suitable color matrixes in accordance with the type of observation region, the type of light source used, or the like, and obtain good color reproduction corresponding to the type of observation region or the type of light source used. This apparatus uses a one-chip color CCD as an image sensing means. A CPU reads out corresponding color matrix coefficients from a ROM, and outputs the coefficients to a color separation circuit. The color separation circuit executes a matrix computation based on the color matrix to convert luminance and color difference signals into primary color signals.