The present invention relates to an apparatus for processing an image signal etc. that are well applicable to removal of coding noise from, for example, an image signal. Based on five consecutive frames of an image signal Va, a memory portion 121 outputs as pixel data xi of predictive taps plural items of pixel data located in a space directional and time directional peripheries with respect to a target position in an image signal Vb. In the case, frames before and after a current frame are subjected to motion compensation by using a motion vector. A class classification portion 124 obtains a class code CL indicating a class to which pixel data of the target position in the image signal Vb belongs, by using the pixel data xi and motion vectors BWV(0), BWV(-1), FWV(0), and FWV(+1). A calculating circuit 126 obtains pixel data y of the target position in the image signal Vb based on an estimation equation by using the pixel data xi and coefficient data Wi that corresponds to the class code CL.