A method and system are provided for controlling the simulated interfacing of a controlled first body, for example a tool, with a second body while providing haptic feedback to a user on such interfacing. Selected representations of the first and second bodies are stored in a memory of a processing apparatus and are used by such apparatus to control simulated movement of the first body relative to the second body in response to a user controlled haptic interface device. Collisions between the first and second body are detected, including the position on each body of each collision, the direction of the collision and the force of the collision, and this information is converted for each collision into a force vector on the first body which is fed back as a corresponding force vector to the interface device, and thus to the user. A feature of the invention is storing a point cloud representation of at least one of the bodies, which point cloud representation is utilized in the simulations. The representations may also be an implicit equation representation, a binary space tree partition representation or the like. A niceness factor may also be utilized in determining force feedback as may a guide zone around at least a portion of one of the bodies which, for example, may be utilized to provide a snap-fit. Other features are also provided.