A system, method and computer program product is provided for interactive user navigation in a real-time 3D simulation. An assembly builder permits a user to build customized physics-based assemblies for user navigation in a variety of virtual environments. These assemblies are stored in a library and are then accessed by a navigation run-time module that runs in conjunction with, or as a part of, a visual run-time application. The navigation run-time module receives high-level user goal requests via a simple and intuitive user interface, converts them into a series of tasks, and then selects the appropriate assembly or assemblies to perform each task. As a result, complex navigation may be achieved. Once selected, an assembly provides a physics-based eye-point model for user navigation. Collisions between the assembly and objects in the simulation are resolved using a real-time physics engine, thus ensuring smooth, cinematic-style eye-point modeling in addition to real-time control.