Further increase of transmission capacity is required for high speed networks and there are some problems to improve WDM system performances. Especially, for-wave-mixing (FWM) is one of the fiber nonlinearities and FWM impairments remain unsolved. In this experiment, we demonstrate digital coherent detection for FWM compensation. Two ASK-modulated optical signals are transmitted through dispersion-shifted fibers (DSF) with an optical amplifier repeater. FWM has been generated by transmitting through DSF. The transmitted signals are combined with phase-locked local oscillator lights because phase information and the phase relation among individual channels can be kept using phase locked light. The detected individual signals are compensated by backward propagation method. The FWM-crosstalk ratio is improved from 12 dB to 17dB and eye-opening is also improved by compensation.© (2010) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.