Systems and methods are described for managing annotations in pen-based computing systems. The systems and methods described herein provide ways to collect, manage, search and share personal information entered by way of handwritten annotations. Annotations are used to drive applications, serve as gestures, find related information and to further manage information. Context information is obtained when a user enters an annotation, and is used to assist in determining and locating relevant content in which the user may be interested, whether in the same document or a different document located on a local computer or on the Internet or other network.