A motion detection device includes a feature extracting section 2 extracting a feature area of an object from among frame images clipped by an image pickup section 1, a feature standardizing section 3 standardizing a size of a feature area extracted by the feature extracting section 2, a balloting section 4 counting up a ballot value of an image area on a frame image located in a standardized pattern that is standardized by the feature standardizing section 3, and a shift speed detecting section 5 calculating a gradient of a counted-up ballot value and calculating a shift direction, a shift speed and position of a feature area based on the gradient.