Because the back-and-forth movement by hand is slower and more strenuous than the leftward-and-rightward swinging, a handheld image-tracking device, which can be applied to an optical mouse or a handheld scanner, uses a rectangle-shaped sensing array to save area of the sensing array and lower down the production cost for the device by reasonably reducing the back-and-forth sensing area for the sensing array; and, so, an amount and time spent for a data comparison in the device is lessened.