A voltage regulator circuit is arranged to provide a regulated output voltage from an unregulated input voltage. Also, the voltage regulator circuit includes a capacitor that is coupled between output and feedback nodes of the voltage regulator circuit. The capacitor may improve transient responses, output noise, and the power supply rejection ratio of the voltage regulator circuit during normal operation. During a start-up of the voltage regulator circuit, the capacitance between the output and feedback nodes is substantially eliminated or substantially reduced in order to reduce the time that it takes for the output voltage to reach a stable voltage level.