A three-dimensional object recognizing system comprises a distance image generating portion for generating a distance image by using image pairs picked up by a stereoscopic camera, a grouping processing portion for grouping the distance data indicating the same three-dimensional object on the distance image, an input value setting portion for setting an area containing distance data group of grouped three-dimensional object on the distance image and also setting input values having typical distance data as elements every small area that is obtained by dividing the area by a set number of partition, a computing portion for computing output values having a pattern that responds to a previously set three-dimensional object by using a neural network that has at least the input values Xin as inputs to an input layer, and a discriminating portion for discriminating the type of the three-dimensional object based on the pattern of the output values.