Techniques for classifying failure messages received from email messages that are sent are provided. A plurality of rules are created that are used to classify a plurality of failure types for email messages. Email messages are sent to an email address associated with an Internet service provider. When an email message fails, a failure message is determined. A rule that applies to the failure message is then determined and based on the rule, a failure type associated with the rule is determined. Once a failure type is determined, an action may be performed based on the failure type. For example, the email address associated with the email message may be marked as invalid.