An electronic system that produces an enhanced spatial television-like audio-visual experience is disclosed. Unlike normal television, the system enables the viewer to control both the viewing direction and relative position of the viewpoint with respect to the movie action. In addition to a specific hardware configuration, this patent also relates to a new video format which makes possible this virtual reality like experience including a system by which multi-channel audio can be recorded in addition to visual information onto a single data channel which is compatible with existing television and audio-visual standards and an optical tracking system to provide both angular and spatial position information to control the display of said imagery and spatial audio.