A web-based hosted solution through which application developers create, manage and monitor application usage analytics in an online manner. Preferably, an application under test is one of: application software, a script-enabled web application, or a rich Internet application (RIA). During the development process, a usage monitoring API is integrated into the application and the application is deployed. As users interact with the application, a log file is generated, typically in one of two ways. If the application is able to write to a local file system (in the user's machine), usage information is gathered in a log file local to the deployed application and then dispatched to an upload server for processing in a batch manner. If the application is not able to write to the user machine's local file system, the usage information is sent to a remote logging server, preferably on a just-in-time basis, and then the log file is generated on the logging server. In either case, preferably the usage information that is tracked comprises "features," "faults" and "failures" of the application, independent of platform, location, and number of deployed application instances.