Maximize multicast transmission throughput for multi-distributed model training tasks

Hanfeng Zhan; Hongli Xu

doi:10.1117/12.2683437

26 June 2023 Maximize multicast transmission throughput for multi-distributed model training tasks

Hanfeng Zhan, Hongli Xu

Proceedings Volume 12721, Second International Symposium on Computer Applications and Information Systems (ISCAIS 2023); 127210F (2023) https://doi.org/10.1117/12.2683437
Event: 2023 2nd International Symposium on Computer Applications and Information Systems (ISCAIS 2023), 2023, Chengdu, China

Abstract

Reducing the significant communication overhead in large-scale distributed model training is a current research hotspot. Some existing studies have reduced the communication cost from workers to parameter servers (upstream) by utilizing the in-network aggregation capabilities of programmable switches or by pruning models to reduce overall communication costs. However, none of them have investigated how to accelerate downlink communication for multiple distributed model training in data centers, given the constraints of programmable switches and links. We propose an algorithm that achieves an approximate ratio of 𝑂(logκ/β + 1). Through simulation, we demonstrate that our algorithm outperforms previous multicast methods.

Citation Download Citation

Hanfeng Zhan and Hongli Xu "Maximize multicast transmission throughput for multi-distributed model training tasks", Proc. SPIE 12721, Second International Symposium on Computer Applications and Information Systems (ISCAIS 2023), 127210F (26 June 2023); https://doi.org/10.1117/12.2683437

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available