A retaining device for a heat sink includes a retaining piece, a retaining arm, a connection seat and a control member. The retaining piece has one end formed with a retaining hole and a top end formed with two tabs. The retaining arm has a vertical surface formed with a retaining hole. The tabs penetrate through the retaining arm and then the connection seat having an accommodating slot. The control member has a disk and a handle connected thereto. The disk in the accommodating slot has a hollow eccentric control slot. A pivot mounted to the tabs may slide in the control slot. The control member is rotated to move the pivot, which causes a relative movement between the retaining piece and the connection seat so that the retaining hole retains a mounting seat of the heat sink while the retaining arm tightly presses against the heat sink.