Heterodyne laser interferometers serve as core instruments for ultra-precision displacement measurement. The main bottleneck restricting their measurement accuracy lies in nonlinear errors (NCE) induced by non-ideal characteristics of optical systems, which are typically manifested in three features: unequal amplitude, DC offset, and non-orthogonality. Based on a rigorous derivation of the series model for nonlinear errors in heterodyne interferometry, this paper compares and analyzes the inherent limitations of traditional analytical compensation and ellipse-fitting methods. It is pointed out that both methods essentially belong to passive mathematical fitting and can hardly eliminate residual harmonic fluctuations under dynamic displacement conditions. To address this issue, an active feedback compensation strategy based on an improved polarizer rotation method is proposed. A closed-loop regulation structure is constructed using a quarter-wave plate and a controllable rotating polarizer. By means of the independent-variable physical freezing mechanism, dynamic nonlinear errors that fluctuate sharply with displacement are equivalently converted into static constant offsets. Combined with static pre-scan stripping and Fast Fourier Transform (FFT), high-precision calibration and initial alignment of the system initial phase offset ??are achieved. Simulation results preliminarily verify the theoretical feasibility of the proposed method under ideal conditions, which can effectively suppress first-order nonlinear errors from the nanometer scale to below 0.1 nm, restore the Lissajous figure to a standard unit circle, and bring the measurement accuracy close to the noise limit of the Cramér-Rao Lower Bound (CRLB). At this stage, this work focuses primarily on theoretical modeling and simulation analysis. This research introduces a new concept for eliminating nonlinear errors at the hardware closed-loop level, and an experimental platform will be constructed for hardware validation in future work, which is expected to offer theoretical and technical support for realizing ultra-precision and high-linearity displacement measurement.