A video detection and monitoring method and apparatus utilizes an application-specific object based segmentation and recognition system for locating and tracking an object of interest within a number of sequential frames of data collected by a video camera or similar device. One embodiment includes a background modeling and object segmentation module to isolate from a current frame at least one segment of the current frame containing a possible object of interest, and a classification module adapted to determine whether or not any segment of the output from the background modeling apparatus includes an object of interest and to characterize any such segment as an object segment. An object segment tracking apparatus is adapted to track the location within a current frame of any object segment and to determine a projected location of the object segment in a subsequent frame.