Delivering a Direct Proof private key in a signed group of keys to a device installed in a client computer system in the field may be accomplished in a secure manner without requiring significant non-volatile storage in the device. A unique pseudo-random value is generated and stored along with a group number in the device at manufacturing time. The pseudo-random value is used to generate a symmetric key for encrypting a data structure holding a Direct Proof private key and a private key digest associated with the device. The resulting encrypted data structure is stored in a signed group of keys (e.g., a signed group record) on a removable storage medium (such as a CD or DVD), and distributed to the owner of the client computer system. When the device is initialized on the client computer system, the system checks if a localized encrypted data structure is present in the system. If not, the system obtains the associated signed group record of encrypted data structures from the removable storage medium, and verifies the signed group record. The device decrypts the encrypted data structure using a symmetric key regenerated from its stored pseudo-random value to obtain the Direct Proof private key, when the group record is valid. If the private key is valid, it may be used for subsequent authentication processing by the device in the client computer system.