The present invention relates to a capsule endoscope system and method of processing image data thereof and is suitable for capturing the inner parts of human body using a capsule endoscope having an internally-usable capsule size and receiving to display image data generated from the capturing. By the present invention, a possible-disease part of a patient is selected and observed through a number of image data received from a capsule endoscope, attribute information containing diagnostic details are supported to facilitate an overall diagnostic result to be interpreted, and internal organs in numerous image data received from a capsule endoscope are easily discriminated from each other.