A system and method for quality-based image compression utilizing adaptively sized blocks and sub-blocks of discrete cosine transform coefficient data and a quality based quantization scale factor is claimed. A block size assignment element in an encoder element selects the block or sub-block of an input block of pixel data to be processed. Blocks with variances larger than a threshold are subdivided, while blocks with variances smaller than a threshold are not subdivided. A transform element transforms the pixel values of the selected blocks into the frequency domain. The frequency domain values are quantized either on a block by block basis or a frame by frame basis utilizing a scale factor that correlates with the quality of the image. The data is then serialized and coded in preparation for transmission.