A novel system and method of video assisted music instrument collaboration over distance. The system and method enable a musician to play a music instrument at one location and have the played music recreated by a music instrument at another location is provided. The system and method can be used to provide distance education for musical instrument instruction and, in this case, each student and instructor of the system has an end point which can connect to other end points in the system to exchange music data, preferably MIDI data, and videoconferencing data through a data network such as the Internet. The system and method can also be used for performances wherein a musician at a first end point plays an instrument and music data, representing the music played, is transferred to a second end point where the music played at the first end point is reproduced and one or more other musicians at the second end point play with the reproduced music in a musical performance. Preferably, each end point includes a music processing engine which buffers data received from another end point to remove the effects of transmission delays and jitter and to discard overly delayed data and to prevent damage to the music instrument at the end point due to undue network delays. Further, the music processing engine can inform the users when network performance is responsible for improper and/or undesired music playback by the instrument at the end point. This buffering by the music processing engine can also allow the synchronization of a video conferencing system between the end points with the playing of music by the instruments at the end points.