Trade-off between Intensity Modulation Direct Detection (IM- DD) and coherent system oriented to the inter-orbit lightwave communication link is performed. Improvement in sensitivity of IM-DD system adopting high power optical fiber booster/pre- amplifiers is discussed preciously. In this paper, IM-DD system with booster/pre-amplifier and five kinds of coherent system (ASK-heterodyne, FSK-heterodyne, PSK-heterodyne, DPSK- heterodyne, and PSK-homodyne) are compared for high-data rate communication. Numerical analysis shows the superior characteristics of coherent communication system in low BER transmission under severe ASE noise.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.