Method and apparatus for handwriting recognition system for ideographic characters and other characters based on subcharacter hidden Markov models. The ideographic characters are modeled using a sequence of subcharacter models and by using two-dimensional geometric layout models of the subcharacters. The subcharacter hidden Markov models are created according to one embodiment by following a set of design rules. The combination of the sequence and geometric layout of the subcharacter models is used to recognize the handwriting character.