Methods, computer code products and devices for encoding and/or decoding video data in multiple passes, the video data having a multiple components each component including multiple coefficients. The method can starting the next pass of the encoding or decoding process immediately after the end of the current encoding or decoding pass for a given component without regard to whether other components have finished the current encoding or decoding pass. In addition, stagger delays and dampers can be used to more closely regulate the encoding or decoding process to ensure that one component is not encoded or decoded too quickly with respect to other components.