A speech recognition interface and computer-readable medium build a grammar for speech recognition that reduces latency in speech-based applications. The interface and medium receive instructions to add a new phrase and semantic information to a grammar. The new phrase is combined with at least one other phrase in the grammar to form a composite grammar structure. The semantic information is then associated with a single word or transition in the grammar structure by selecting the first possible word or transition that semantically differentiates the new phrase from all other phrases in the grammar structure. By placing the semantic information in this position, the semantic information is placed as far forward in the grammar as possible without introducing semantic ambiguity into the grammar structure.