A passive sound telemetry system and method is provided, in which phase differences are calculated for every pulse from acoustic signals received via three or six acoustic receivers, to measure a distance and angle with respect to a sound source. The system is applied to an operating toy giving chase to a user while following the user command. The telemetry system includes first to fourth acoustic receivers (111 114) contained in the housing for receiving first to fourth acoustic signals from a user, first to fourth zero crossing detectors for generating first to fourth pulse signals based on the first to fourth acoustic signals, earliest reception signal detector for detecting a pulse signal having the earliest phase among the first to fourth pulse signals, first to fourth arrival time difference calculator (70) for calculating arrival time differences between the pulse signal of the earliest phase and the first to fourth pulse signals, respectively, thereby obtaining an angle and a distance between the center of acoustic receivers and the user, based on a time difference between the second and third minimum arrival time difference values.