A game apparatus allows a virtual object to act in a mixed reality space as if it had its own will. A player can play a game with the virtual object. Rules for controlling the action patterns of the virtual object on the basis of the objective of the game, and the relative positional relationship between the virtual object and the real object is pre-stored. The next action pattern of the virtual object is determined based on an operator command, the stored rule(s), a simulation progress status, and geometric information of a real object(s).