A quantization matrix adjusting method for avoiding underflow of VBV by enlarging the default quantization matrix. The adjusting method comprises the steps of: updating a used bits BB_X and an average quantizer scale Avg_Q; allocating the bit budget BB_C for the current picture; calculating an estimated quantizer scale EstQ_C according to the used bits BB_X, the average quantizer scale Avg_Q and the bit budget BB_C; deciding the operation mode, when the estimated quantizer scale EstQ_C is larger than a threshold Th_Q, assigning the operation mode as an alert mode, otherwise remaining at normal mode; enlarging the default quantization matrix as a new quantization matrix when the operation mode is the alert mode; and coding the current picture using the new quantization matrix and default quantization matrix in the alert mode and normal mode, respectively.