There is provided a scene change detection method which is independent from an encoder of a file, and can assure high real-time performance and effective, quick processes, and an image processing apparatus that can implement the method. Frame image data is extracted from moving image data (S11), the frame image data is segmented into a plurality of blocks (S12), labels are assigned in accordance with feature amounts acquired in units of blocks (S13), a sequential label set is generated by arranging the assigned labels in a predetermined block order (S14), similarities between the generated sequential label set and sequential label sets of a previous frame image data group are computed by two-dimensional DP matching and a scene change frame in the moving image is detected from a computed similarity group (S15, S16), and the detected scene change frame information is stored in correspondence with the frame image data.