The invention provides techniques for secure messages transmission using a public key system to exchange secret keys. A first entity creates public and private keys by generating a product n of two large, randomly chosen prime numbers, and then generating random matrices {A, C}, in the group GL(r,Z.sub.n) with a chosen matrix rank r such that AC is not equal to CA, and then generating a matrix B=CAC, and finding a matrix G that commutes with C. Matrices A, B, G and the integers n and r are then published as the public key and matrix C is then kept as the private key. A second entity then obtains the public key and calculates a secret matrix D that commutes with G, and further calculates the matrices K=DBD and E=DAD. The message to be sent is then encrypted using matrix K as the secret key and then sent to the first entity with matrix E. First entity then retrieves secret matrix K using K=CEC and then decrypts the received encrypted message using the retrieved secret matrix K.