Systems and methods for filtering and viewing collaborative indexes of annotations of multimedia or video streams can include obtaining a multimedia or video stream, obtaining user indexes of annotations of the multimedia or video stream, merging the indexes, associating portions of the multimedia or video stream with the annotations, determining high-value annotations and displaying the high value annotations. Systems and methods for providing useful visualizations of the indexes of annotations and high value annotations are also provided.