A system and method for classifying a voice signal to one of a set of predefined categories, based upon a statistical analysis of features extracted from the voice signal. The system includes an acoustic processor and a classifier. The acoustic processor extracts features that are characteristic of the voice signal and generates feature vectors using the extracted spectral features. The classifier uses the feature vectors to compute the probability that the voice signal belongs to each of the predefined categories and classifies the voice signal to a predefined category that is associated with the highest probability.