A parallel processing apparatus capable of flexibly solving at a high speed the problem of synchronization wait when a plurality of tasks are generated, wherein a processor element PE12 specifies pipe counters and calls up a plurality of tasks with respect to processor elements PE13 to PE15 and waits for synchronization according to need by a synchronization wait command, an arbiter 56 increases a count value of a corresponding pipe counter when a task is called up and decreases the count value of the corresponding pipe counter when the task is ended, and the processor element PE12 compares the count value contained in the synchronization wait command and the count value of the corresponding pipe counter of the arbiter 56 when executing the synchronization wait command, releases the synchronization wait when they coincide, and enters the synchronization wait when they do not coincide.