A compiler comprises a lexical scanner with a dynamic character identification table that accommodates different standards for encoding characters in source and target characters. The scanner is associated with data files that enable conversion of the character identification table and also source code to conform to different standard character set representations. The scanner compares the character sets associated with the source and target codes. The source code is loaded into a buffer, and if a discrepancy in character sets exists, the buffered source code is converted to the target character set. The scanner also compares the character sets associated with the target code and the character identification table. If a mismatch exists, the scanner converts the character identification table to conform to the target character set. The lexical scanner provides greater flexibility in reading and generating program code.