Annotating activities are associated with portions of recordings by activity/recording data such as a table in which annotation identifiers are paired with timestamps. To obtain activity/recording data, an image input device, such as a manually positionable scanner or a fixed position camera, can produce an image signal set with information about a manual annotating activity and a recording/playback device can provide a recording portion signal, such as a timestamp, with information about a portion of a recording. The image signal set and the recording portion signal can then be used to automatically obtain the activity/recording data, associating information about the manual annotating activity with information identifying the portion of the recording. The information about the manual annotating activity can be an annotation identifier that includes information about the position or shape of the annotation. The activity/recording data can also include, for each annotation identifier, an image of the annotation. Activity/recording data can be used to playback a portion of a recording. Another image signal set can be obtained with information about a manual request activity, for example, and if the manual request activity is sufficiently similar to a manual annotating activity, the associated portion can be played back. Or a user can provide a selection signal indicating one of the annotations to obtain playback of the associated portion of the recording. If the annotating activity was performed with a keyboard or other user input device, the annotations can be printed, and the user can use a hand-held scanner to select an annotation to obtaining playback.