A conditional access scrambling or encryption technique for a packetized digital data stream, e.g., a MPEG-2 bitstream, (1) by scrambling or encrypting the data payload of selective transport payload packets (e.g., every nth packet); (2) by scrambling or encrypting only a portion (e.g., a central portion) of the data payload of every transport payload packets; or (3) by both scrambling or encrypting the data payload of every nth packet and scrambling or encrypting only a portion of the data payload of every transport payload packet. With advance knowledge by the subscriber's descrambler of which payload packets are being scrambled, and/or of which portion of which payload packets, descrambling can be performed with minimal processing and without the need to first recreate the underlying image and/or audio.