A fixing device and an image forming apparatus using this, in which even if permanent warping is produced on a roller member that forms a fixing nip portion, no collision sound is produced due to the collision of gears, the fixing device including: two roller members that form a nip portion in which a recording medium is pressed by pressure means and transported; and a drive gear that engages with a shaft portion of one of the two roller members and transmits driving force to that roller member. A key is formed in the drive gear and a key groove having a gap opened therein is formed in the shaft portion, and when a rotation velocity of the roller member increases, transmission of driving force from the drive gear to the shaft portion is suspended and the drive gear slides on the shaft portion.