A method and apparatus for performing speech recognition using observable relationships between words. Results from a speech recognition pass can be combined with information about the observable word relationships to constrain or simplify subsequent recognition passes. This iterative process greatly reduces the search space required for each recognition pass, making the speech recognition process more efficient, faster and accurate.