Data rate smoothing is disclosed. In one embodiment, a system includes a real-time encoder and a rate-smoothing mechanism. The real-time encoder has a first layer and a second layer, where the first layer sends packets of data to the second layer at a dynamic bit rate. The mechanism adjusts the dynamic bit rate such that the rate is less than a predetermined maximum rate. In a further embodiment, the second layer also sends the packets to the mechanism, and the system also includes a buffer and a client. The buffer temporarily stores the packets received by the mechanism. The client receives the packets from the mechanism, where the mechanism deletes packets from the buffer when a predetermined backlog threshold is exceeded.