With an image-based rendering (IBR) system based on linear interpolated concentric mosaics, an observer is able to wander through a real or synthesized scene and have new view images (optionally including stereo images) of the scene rendered as the observer moves. According to one embodiment, multiple viewing rays are extended in a viewing direction from the image to be rendered at a view point. For viewing rays that do not coincide with captured images, an image is generated based on the captured images. The image is generated by interpolating between at least two captured images based on a constant distance to objects in the scene. The view images that are displayed can be mono or stereo (e.g., simulating a pair of eyes of the observer).