An automated testing framework enables automated testing of complex software systems. The framework can be configured for test selection, flow definition, and automated scheduled testing of complex computer systems. The framework has facilities for result analysis, comparison of key performance indicators with predefined target values, and test management.