An apparatus for and a method of audio recording in packet based telephony systems. Using the present invention, the equivalent of four audio channels are recorded utilizing only two recording channels. Each channel recorded comprises the stream of packets generated and transmitted by each endpoint to the other side. The RTP packets include the samples generated by the particular endpoint in addition to the timestamp of the samples received from the other side actually played by the endpoint. The recording device has knowledge of what was played at the other endpoint in order to accurately playback the audio samples generated by and received from the other endpoint. The recording device receives a packet stream containing the audio generated on each endpoint and the timestamp of the packet from the other side that was played on the endpoint. The recording device can reconstruct from this data the audio signal that was actually played on each endpoint.