A hardware structure for adaptively prediction coding an image signal using spatial correlation and a method therefor are provided. The prediction coding apparatus has a packet change detection unit which determines whether or not blocks used to determine a prediction direction for a current block for which prediction is currently performed is included in a same packet to which the current block belongs; a prediction reference value providing unit which provides a prediction reference value used in prediction according to the result of determination by the packet change detection unit; and a prediction calculation unit which outputs a predictive coded value or a predictive decoded value using the prediction reference value provided by the prediction reference value providing unit and the DCT component of the current block according to an operation mode. According to the apparatus and method, the size of a prediction apparatus is reduced, and one apparatus is used both in coding and decoding.