In the last decade, the success achieved by convolutional neural networks (CNN) has changed the trend of research in the computer vision field from manual features to learned features for various applications. However, designing the architecture of CNN and selecting optimum values of its hyperparameters poses a critical challenge because of their massive and complex search space. Automatically picking up the optimal values of hyperparameters may facilitate the designers of CNN-based deep learning models. Swarm intelligence (SI)-based improved competitive gray wolf optimizer (ICGWO) in continuous search space is proposed for optimizing hyperparameters of custom-designed CNN architecture. To evaluate the performance of the ICGWO, we employ 16 benchmark functions. We also assess the effectiveness of the ICGWO by evaluating it with six state-of-art meta-heuristic algorithms. The experiment results show that the ICGWO can obtain the best accuracy on eleven benchmark functions and second best on two benchmark functions. The optimized CNN recognizes static signs of English alphabets in Indian Sign Language (ISL). We also evaluated the optimized CNN on the publicly available ISL dataset for the English alphabet, where it attains 99.93% validation accuracy. In order to evaluate the efficacy of the proposed variant of CGWO, we optimized the hyperparameters of CNN using the existing SI-based search optimization algorithms namely particle swarm optimization, whale optimization algorithm (WOA), differential evolution, and gray wolf optimizer as well.