Open Access
16 March 2023 Hybrid synthetic data generation pipeline that outperforms real data
Sai Abinesh Natarajan, Michael G. Madden
Author Affiliations +
Abstract

Fine-tuning a pretrained model with real data for a machine learning task requires many hours of manual work, especially for computer vision tasks, where collection and annotation of data can be very time-consuming. We present a framework and methodology for synthetic data collection that is not only efficient in terms of time taken to collect and annotate data, making use of free- and open-source software tools and 3D assets but also beats the state-of-the-art against real data, which is the ultimate test for any similar-to-real approach. We test our approach on a set of image classes from ObjectNet, which is a challenging image classification benchmark test dataset that is designed to be similar in many respects to ImageNet but with a wider variety of viewpoints, rotations, and backgrounds, which can make it more difficult for transfer learning problems. The novelty of our approach stems from the way we create complex backgrounds for 3D models using 2D images laid out as decals in a 3D game engine, where synthetic images are captured programmatically with a large number of systematic variations. We demonstrate that our approach is highly effective, resulting in a deep learning model with a top-1 accuracy of 72% on the ObjectNet data, which is a new state-of-the-art result. In addition, we present an efficient strategy for learning rate tuning that is an order of magnitude faster than regular grid search.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Sai Abinesh Natarajan and Michael G. Madden "Hybrid synthetic data generation pipeline that outperforms real data," Journal of Electronic Imaging 32(2), 023011 (16 March 2023). https://doi.org/10.1117/1.JEI.32.2.023011
Received: 26 July 2022; Accepted: 14 February 2023; Published: 16 March 2023
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
KEYWORDS
Lawrencium

3D modeling

Data modeling

Education and training

Machine learning

Light sources and illumination

3D image processing

Back to Top