An apparatus and method for configuring a gateway device for a modem over packet network session is disclosed. For example, a system may include a first gateway device coupled to a first client device and a second gateway device coupled to a second client device, where the first gateway device is coupled to the second gateway device over a packet network. The trans-compression types and capabilities of the first and second gateway devices are determined and used to generate a set of negotiation parameters. The first gateway device then negotiates with the first client device using the negotiation parameters to select a first compression algorithm. A second compression algorithm selected between the second gateway device and the second client devices is also determined by the first gateway device. Then, the trans-compression mode of operation of the first gateway device is configured based on the trans-compression types and capabilities of the first and second gateway devices and the selected first and second compression algorithms. The second gateway device may be similarly configured.