A video signal processing apparatus which is provided for a digital video decoder and can automatically and properly adjust a fluctuation of the number of sampling data of digital composite video signals includes: an adjusting circuit which performs a partial adding/deleting process to the video signals so as to correct the fluctuation in number of sampling data constructing the digital composite video signals supplied to the decoder; and a discriminating circuit which compares the number of sampling data of each video signal and the preset reference number of sampling data and outputs an operation signal for the adding/deleting process to the adjusting circuit when a difference between them as a comparison result lies within a predetermined adjusting range.