An electron beam inspection system of the image projection type includes a primary electron optical system for shaping an electron beam emitted from an electron gun into a rectangular configuration and applying the shaped electron beam to a sample surface to be inspected. A secondary electron optical system converges secondary electrons emitted from the sample. A detector converts the converged secondary electrons into an optical image through a fluorescent screen and focuses the image to a line sensor. A controller controls the charge transfer time of the line sensor at which the picked-up line image is transferred between each pair of adjacent pixel rows provided in the line sensor in association with the moving speed of a stage for moving the sample.