A system and method for automatic insertion of enhanced personal address book contact data from the telephone. The PAB telephone network-based system includes an application unit to coordinate the addition of new contact data by a user via his telephone, and/or is used in conjunction with his Web-browser or other Internet/Intranet access device. The system includes an ASR recognition server used to assist in adding a new contact to the PAB, and also to match the spoken name gathered with the possible data augmented data as extracted from a system-resident white pages, or other source, such as an external white pages. The system is connected to a telephone network, or group of telephone networks including elements from one or a combination of components of E1, T1 or VoIP apparatus. When at least one name matching the new contact telephone number is found, the system attempts to automatically confirm it or choose between the names based on matching the spoken name and the textual name found using ASR. The appropriate matching name and associated data from the White pages are inserted in the PAB of the user as a new contact entry. The augmented text name of the contact can be used by the ASR subsystem to replace the original training or can be used in addition to the original training.