Optical minimum-shift keying (MSK) format has constant envelope and continuous temporal phase shift, and exhibits a promising application for its high compact spectrum. Coherent detections have been proved to be effectively for the significant improvement of receiver sensitivity. In this paper, a heterodyne detection scheme to demodulate 40Gb/s MSK signal is proposed, and simulation analysis is given to the receiver sensitivity, the laser line-width and frequency offset. Comparison is drawn between the presented heterodyne detection and homodyne detection, and investigation is made on the performance of the receiver in different schemes.© (2010) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.