Provided are a method and apparatus for compensating for an acceleration error in an inertial navigation system (INS). The apparatus includes a motion detector that detects the motion of the INS and outputs information on stationary time intervals during which the INS is stationary and a moving time interval during which the INS is moving; a sensor portion that measures accelerations of the INS for the stationary time intervals and moving time interval, respectively; an error model determiner that determines an acceleration error corresponding to the moving time interval using accelerations measured during the stationary time intervals and outputs an acceleration subjected to correction of the acceleration error for the moving time interval; and a position calculator that integrates the corrected acceleration for the moving time interval and outputs the position of the INS.