Described is a technology by which software instrumentation data collected
during software program usage sessions is analyzed to identify potential
problems with software program usage, such as based on frequency of
problem occurrence during the usage sessions. Reliability metrics may be
calculated from the information. Failure data additionally collected
during the usage sessions may be accessed to derive details that
correspond to the potential problems. In one example, the information may
be analyzed to determine which alerts and/or asserts occurred most often,
and/or to determine a relationship between user interface control
operations (e.g., clicks and usage of commands) and alerts or asserts.