The semiconductor industry continuously moves toward smaller features and more complex, three-dimensional (3D) structures to enable next-generation devices. Thus there is a high need for 3D metrology techniques that would provide feedback to the advancing nanoscale fabrication processes. We introduce an approach for evaluating, calibrating, and developing new metrology of nanopatterns through comparing the new metrology data to 3D ground truth data, obtained by accurate scanning transmission electron microscopy (STEM) tomography 3D characterization. We demonstrate this approach by evaluating 3D height maps of sub-20-nm patterns obtained using multi-secondary electron detector scanning electron microscopy (multi-perspective SEM) and photometric stereo algorithm. We demonstrate the importance of full 3D characterization, including cross-sectional structure, height fluctuations, and line edge roughness to truly probe the pattern’s average structures and their 3D variations. Although STEM tomography is a high-resolution and accurate method, it is considered demanding in terms of sample preparation and cannot be used as an efficient in-line metrology method. However, we show that it can be used as a reliable 3D characterization technique to obtain the necessary 3D data for the development of new 3D metrology tools and demonstrate this approach by evaluating the accessible, in-line, multi-perspective SEM.