A method of and apparatus for setting a virtual image region is provided in order to efficiently perform motion estimation and compensation of a panorama image including 360-degree omni-direction image information. The method includes the steps of searching a previous panorama image for reference data corresponding to all or part of a virtual image region of a current patch desired to be motion-estimated in order to determine the presence of the reference data, and selectively mapping the reference data to all or part of the virtual image region. By doing so, the panorama image containing omni-directional image information is coded by using a 2-dimensional moving picture coding method without change, while the coding efficiency in the process of motion estimation and compensation can be greatly improved.