The present invention relates to an image scrambling method and apparatus. The apparatus comprises a first dividing decoder for dividing the MPEG encoded image data stream into a plurality of data segments (for example, macro blocks or slices); at least one arrangement table recording arrangement information to be used for arranging the data segments; an arrangement circuit for arranging the data segments output from the first dividing decoder in accordance with the information recorded in the arrangement table to output a scrambled image data stream; a second dividing decoder for dividing the scrambled image data stream into a plurality of data segments; at least one rearrangement table recording rearrangement information to be used for rearranging the data segments output from the second dividing decoder; and a rearrangement circuit for rearranging the data segments output from the second dividing decoder in accordance with the rearrangement information recorded in the rearrangement table.