An electronic input device such as an electronic pen is provided to annotate a paper document. The input device records an annotation and an image of human-comprehensible content in the document sufficient to identify the document and possibly a location in the document. The human-comprehensible content is used to locate a digital version of the document and determine a corresponding location of the annotation in the digital version of the document. A computer system such as a server system may receive and store the annotation in association with the digital version of the document. The server system may further augment the digital version of the document with the annotation and send the augmented version to an output device for display and/or printing.