Conventional parameter calibration algorithms for InSAR system are based on calibrators’ absolute height, so absolute height error is a key factor that influences the calibration results. Since current height measurements are affected by reference horizon and measure environment, accurate measurement is much more costing and difficult, but accurate height difference of calibrators within sight is much easier to measure with low cost, so calibrators’ height differences are used to calibrate system parameters in this paper. It not only avoids the measurement of calibrators’ absolute height, but also reduces the condition number of sensitivity matrix. Basic criteria for locating calibrators are also given. The simulation results show the effectiveness of this calibration method, and also reconstruct an accurate relative height terrain.