A method and apparatus for filtering an input text stream includes
receiving a definition of a filter configuration and modifying the input
text stream according to the filter configuration so as to generate a
filtered text stream. The filtered text stream includes positioning
information for the input text stream. The positioning information may be
useable by a downstream scanning device of a parser.