A method and apparatus are described for performing header-based scaling and cropping of images compressed using multi-scale transforms. In one embodiment, the method for adaptively scaling an image comprises receiving a file containing a header that contains multi-scale entropy distribution information on blocks of an image; for each block, determining that the block retains significance at a scale upon determining that an entropy of a multi-scale coefficient of a block at the scale is greater than a mean entropy of multi-scale coefficients of blocks in at least one coarser scale; and scaling the image to a coarsest scale at which a threshold percentage of the blocks retain significance at the scale.