An image data compression device includes a quantization part quantizing image data with a quantization step that varies based on a quantization parameter, a FIFO buffer part buffering quantized data of a plurality of frames, a coded data formation part reading out the quantized data from the FIFO buffer part asynchronously with a writing to the FIFO buffer part and generating coded data by encoding the quantized data, a rate control part controlling a data size of the coded data by changing the quantization step and a frame skip part skipping a generation process of the image data. The rate control part calculates the quantization parameter by using a predicted data size of the coded data of a previous frame which is calculated from a data size of the quantized data of the previous frame. The frame skip part performs the skip process if a frame, in which the quantization parameter becomes larger than a skip threshold, appears consecutively in a number of times which is equal or more than the number of a skip succession threshold.