Conversations that take place over an electronically recordable channel are analyzed by constructing a set of features from the speech of two participants in the conversation. The set of features is applied to a model or a plurality of models to determine the likelihood of the set of features for each model. These likelihoods are then used to classify the conversation into categories, provide real-time monitoring of the conversation, and/or identify anomalous conversations.