A vision system and methods to detect rogue objects on a product processing line are disclosed. The vision system includes a camera, a source of illumination, a user interface, and a computer-based platform including a memory. The vision system captures images from objects on the product processing line as the objects pass by the camera which is mounted on the product processing line. Image pixel data is converted to vectors of mean pixel values for each captured image from each object, using the vision system. The vectors are compared to a vector training set stored within the memory of the vision system to determine if the objects correspond to "rogue" objects or not, using the vision system. The vector training set is representative of the current product (i.e., objects) being processed on the product processing line.