A proxy engine (108), in communication with a web crawler (100),
extends the performance of the web crawler (100) by modifying hyperlink
requests and creating synthetic hyperlink requests from data received from a web
site (104). The proxy engine (108) converts hyperlinks to a method
used by a target web site. The proxy engine receives data from a web site (104)
located across a network (102), and then determines whether additional data
from the web site (104) is extractable. In response to determining that
additional data from the web site (104) is extractable, the proxy engine
(108) creates at least one synthetic hyperlink for extracting the data from
the web site (104). The proxy engine (108) then combines the at least
one synthetic hyperlink with the data received from the website (104) to
create combined data and then sends the combined data to the crawler (100).