Techniques for indexing multimedia data substantially concurrently or contemporaneously with its capture to convert a real world event into an accessible database in real time are provided. The present invention introduces a new paradigm of converting a real world event in real time into a rich multimedia database by processing data from multiple sensors observing the event. Real time analysis of the sensor data, coupled with domain knowledge, results in instant indexing of multimedia data at capture time itself. This yields the semantic information to answer complex queries about the content, and the ability to extract portions of data that correspond to complex actions performed in the real world.