When a pressure of fuel supplied to an engine is higher than a target, the fuel is carried in a reverse direction by rotating a fuel pump in reverse, and in accordance therewith, the fuel pressure is made to fall with good responsiveness.