The non-reciprocal and dynamic nature of polarization mode dispersion (PMD) in optical fibers can be a problem for accurate time and frequency transfer. Here, a simple, passive solution is put forward that is based on transmitting optical pulses with alternating orthogonal polarization. The fast and deterministic polarization modulation means that the PMD noise is pushed far away from the frequencies of interest. Furthermore, upon reflection from a Faraday mirror at the receiver, the pulses have a well-defined polarization when they return to the transmitter, which facilitates stable optical phase detection and fibre phase compensation. In an open-loop test setup that uses a mode-locked laser and a simple pulse interleaver, the polarization mode dispersion is shown to be reduced by more than two orders of magnitude.