A method and apparatus is disclosed herein for generating visualizations of documents based on semantic information. In one embodiment, the method comprises selecting a plurality of image segments having a plurality of attributes that includes a semantic importance information for each of the plurality of document image segments and determining a plurality of positions and a plurality of geometric operations to be performed on the plurality of document image segments according to the plurality of attributes and display constraints.