The invention described herein provides a method and apparatus for document processing that efficiently separates and interrelates single modalities, such as text, handwriting, and images. In particular, the present invention starts with the recognition of text characters and words for the efficient separation of text paragraphs from images by maintaining their relationships for a possible reconstruction of the original page. The text separation and extraction is based on a hierarchical framing process. The process starts with the framing of a single character, after its recognition, continues with the recognition and framing of a word, and ends with the framing of all text lines. The method and apparatus described herein can process different types of documents, such as typed, handwritten, skewed, mixed, but not half-tone ones.