A frequency-converting circuit and a down converter with the frequency-converting circuit are disclosed. The above-mentioned frequency-converting circuit is used for converting an RF signal into a first baseband signal according to a poly-phase LO signal. The frequency-converting circuit includes a coupler, a first transduction unit and a first switching unit. The coupler is for receiving and splitting the RF signal and delivering a first RF signal via the first output terminal thereof. The first transduction unit is for amplifying the first RF signal. The first switching unit is for performing switching operations on the output signal of the first transduction unit and producing the first baseband signal.