A system and method are provided for calibrating orthogonal polarity in a
multichannel optical transport network (OTN) receiver. The method accepts
a composite signal and separates the polarization of the signal into a
pair of 2.sup.n-phase shift keying (2.sup.n-PSK) modulated input signals
via Ix and Qx optical signal paths, where n.gtoreq.1. Likewise, a pair of
2.sup.p-PSK modulated input signals are accepted via Iy and Qy optical
signal paths where p.gtoreq.1. Polarization-adjusted I'x, Q'x, I'y, and
Q'y signals are generated. An average magnitude is compared to either
2.times.the absolute magnitude of (I'x and Q'x), or 2.times.the absolute
magnitude of (I'y and Q'y). The average magnitude value can be used that
is either 2.times.(a predetermined peak signal amplitude), or the sum of
the absolute magnitudes of (I'x and Q'x) and (I'y and Q'y). The
polarization-adjusted I'x, Q'x, I'y, and Q'y signals are modified until
the magnitude comparison is about zero.