The present invention is directed to gesture-based reporting system and method, which includes a client computer with high-resolution image displaying device, and an input device which is a programmable stylus, where the predetermined information contained within the report is defined by a series of symbols or gestures, which are drawn directly onto the image displayed on the image displaying device, using the programmable stylus. The gestures or symbols used, utilize an economy of symbols that are diverse in nature and have broad based appeal to the population of end users. At the same time, they can be made applicable to a variety of different specific situations, modalities, pathologies, etc., in order to interpret the imaging study. Therefore, unlike a traditional text report (where the image and text report are distinct and separate from one another), the informational content contained within the image and the gesture-based report are inseparable.