When encoding and decoding bit planes, a decision is made in the clean up pass if zero coding or run length coding should be performed. Embodiments of the invention provide a zero coding or run length coding decision instruction. The instruction will determine whether significance state variables associated with selected coefficients bits and their immediate neighbors are zero. If all the significance states are determined to be zero, then run length coding is performed. Else, zero coding is performed.