A method models a set of trajectories of a set of objects; there is one trajectory for each object. A sequence of positions of each object is obtained over a duration of time to define a trajectory of the object. The sequence of positions is then projected to a hidden Markov model to model the trajectory of each object.