An automated method and system for testing a web site for vulnerability to a cross site scripting (XSS) attack are disclosed. The automated tool injects a tracer value into both GET and POST form data, and monitors the resultant HTML to determine whether the tracer value is returned to the local machine by the server to which it was sent. If the tracer value is returned, the automated tool attempts to exploit the web site by injecting a non-malicious script as part of an input value for some form data, based on the location in the returned HTML in which the returned tracer value was found. If the exploit is successful, as indicated by the non-malicious script, the automated tool logs the exploit to a log file that a user can review at a later time, e.g., to assist in debugging the web site.