An inverse-modified discrete cosine transform and overlap-add method, and hardware structure for MPEG Layer3 audio signal decoding. In order to have the MPEG Layer3 audio signal decoder have more competitive power in the consumer market, the present invention provides a low cost fast algorithm of the inverse-modified discrete cosine transform and overlap-add, so that the quantity of the operation needed in the decoding process can be significantly reduced to enhance the system performance. Afterwards, according to the fast algorithm, the present invention provides a hardware structure that is suitable for the inverse-modified discrete cosine transform and overlap-add in the MPEG Layer3 decoder. Since the hardware structure of the present invention makes the MPEG Layer3 decoder able to be implemented by the application specific integrated circuit (ASIC), the entire system can fulfill the low cost and high performance requirements.