A method for dynamic scene modeling and change detection applicable to motion analysis utilizes optical flow for capturing and modeling the dynamics of the scene. Uncertainties in the measurements are evaluated and utilized in order to develop a robust representation of the scene in a higher dimensional space. In another embodiment, a dynamical model of the scene is developed that utilizes multiple past frames to predict the next frame. Incremental methods for updating the model are developed and, towards detection of events, a new measure is introduced that is based on a state-driven comparison between the prediction and the actual observation.