A speech recognizer has an acoustic model that includes word-specific models, that are specific to candidate words. The candidate words would otherwise be mapped to a series of general phones. A decoder transcribes input speech into words formed by shared phones, word-specific phones and a combination of shared word-specific phones.