A system and method for selectively monitoring, recording, storing, and handling telephone conversations through the use of speech analysis is disclosed. In particular, the present invention utilizes a speech analyzer to analyze a speech signal during a telephone conversation between two parties, and a recording and storage device to record and store the telephone conversation. Based on variations in signal characteristics related to the emotional state of the caller, the system selectively generates a trigger to keep the stored recording of the telephone conversation. The present invention also selectively determines whether to send a notification in response to said trigger.