A method for processing speech audio in a network connected client device can include selecting a speech grammar for use in a speech recognition system in the network connected client device; characterizing the selected speech grammar; and, based on the characterization, determining whether to process the speech grammar locally in the network connected client device, or remotely in a speech server in the network. In one aspect of the invention, the selecting step can include establishing a communications session with a speech server; and, querying the speech server for a speech grammar over the established communications session. Additionally, the selecting step can further include registering the speech grammar in the speech recognition system.