An operation recognition system including an object to be operated, at least one camera, a processor, and a controller. At least one camera is configured to acquire image data of an operator in a predetermined period of time, and a processor is configured to perform recognition processing on the acquired image data. Upon processing the image data, the processor is configured to define a virtual plane in between the object to be operated and the operator. The processor is configured to then perform a process to determine if a predetermined part of the operator penetrates the virtual plane based on the recognition processing. Based on this determination, the controller is configured to control the object to be operated.