A method and system for incrementally processing program annotations is provided. A source model, a doclet model and a semantic model are used to represent and transform annotations incrementally. The source model represents actual types, fields, and methods that contain annotations. The doclet model includes doclet objects representing tags and attributes for a type, field, and method for a given namespace. Tag handlers registered for the given namespace detect changes in annotations and create or update doclet objects based on the changes. Changes in the doclet model are then transformed into the semantic model and may be persisted for restoration.