A transmitting and receiving apparatus and method for optimizing performance of an adaptive modulation and coding (AMC) in a multiple input and multiple output antenna (MIMO) communication system. When the AMC is applied to the MIMO system, the optimal scheme is different depending on the MIMO channel situation, the maximum transmission power, and the maximum modulation order. The transmitting apparatus includes an ordering selector that selects a successive interference cancellation (SIC) scheme in order to obtain a maximum MIMO-AMC performance, and the receiving apparatus includes a channel quality information (CQI) generator corresponding to the SIC scheme selected at the transmitting apparatus.