The present invention aims at the production of musical sounds by calculating motion data based on inputted image data using a simple technique without preliminarily preparing playing information or the like and by producing musical sounds based on the calculated data. A musical sound producing apparatus includes an operation part specifying means which extracts motion data indicative of motions from differentials of respective pixels corresponding to image data of a plurality of frames using image data for respective frames as an input; a musical sound producing means which produces musical sound data containing a sound source, a sound scale and a sound level in accordance with the motion data specified by the motion part specifying means; and an output means which outputs the musical sound data produced by the musical sound producing means, wherein an image database in which patterns are registered and an image matching means are provided, and a musical sound synthesizing means is provided to the musical sound producing means, in the musical sound producing means, so as to synthesize the musical sound data with other sound data, thereby producing the musical sound data.