A sports simulation system includes a projectile tracking apparatus having a display surface on which a visually apparent three-dimensional sports scene is presented. The projectile tracking apparatus captures images of a projectile tracking region disposed in front of the display surface to detect a launched projectile traveling through the projectile tracking region towards the display surface. At least one processing stage communicates with the projectile tracking apparatus and is responsive to the image data to determine the three-dimensional positions, velocity and deceleration/acceleration of a detected projectile traveling through the projectile tracking region. The determined three-dimensional positions, velocity and deceleration/acceleration are used by the at least one processing stage to calculate a trajectory of the launched projectile into the visually apparent three-dimensional sports scene. Updated image data is generated by the at least one processing stage that includes a simulation of the launched projectile into the visually apparent three-dimensional sports scene following the calculated trajectory. A display device coupled to the at least one processing stage receives the image data from the at least one processing stage and presents the visually apparent three-dimensional sports scene, including the simulation, on the display surface.