This invention discloses an image processing apparatus which divides moving image data at appropriate positions and sets key frames at appropriately distributed positions in the entire moving image. This image processing apparatus records action information (e.g., a zoom action) associated with actions made during image sensing and environment information (e.g., rotation information due to a pan action) associated with an image sensing environment during image sensing as additional information of the moving image data. Then, the apparatus divides moving image data for one shot into a plurality of sub-shots (S807, S811) in accordance with at least one of the recorded action information and environment information (S802-S806, S808-S810).