Reliable phase-only spatial light modulators (SLMs) are in demand for accurate phase modulation in a wide range of fields. Due to the nonlinear optical response of liquid crystals and the limited manufacturing process available, the spatial nonuniformity of the phase modulation by the pixels should be measured and/or calibrated. We propose an in situ calibration method based on digital holography to calibrate the spatial nonuniformity of phase modulation of the SLM. The SLM panel is divided into blocks composed of pixels. The differential phase on hundreds of blocks can be reconstructed through the holograms. The distribution of modulated phase can then be derived after eliminating statics phase anomalies. The spatial nonuniformity of the panel can be measured for calibration with high efficiency. A modulated phase step on the SLM was calibrated to increase linearly. The spatial nonuniformity was calibrated to decrease by more than 75% using only a beam splitter and an imaging sensor. The in situ strategy for low cost and efficient calibration was demonstrated with optical experiments using a 4K (3840  ×  2160  pixels) phase-only SLM.