A Dynamic Performance-Based Flow Control Method for High-Speed Data Transfer

• Published in: IEEE transactions on parallel and distributed systems Vol. 21; no. 1; pp. 114 - 125
• Main Authors: Eckart, B., Xubin He, Qishi Wu, Changsheng Xie
• Format: Journal Article
• Language: English
• Published: IEEE 01.01.2010
• Subjects: Adaptive control; Bandwidth; Buffer overflow; bulk transfer; Control systems; Delay; Flow control; Hardware; high-speed protocol; Mathematical model; Programmable control; Protocols; Prototypes; reliable UDP
• ISSN: 1045-9219; 1558-2183
• DOI: 10.1109/TPDS.2009.37

New types of specialized network applications are being created that need to be able to transmit large amounts of data across dedicated network links. TCP fails to be a suitable method of bulk data transfer in many of these applications, giving rise to new classes of protocols designed to circumvent TCP's shortcomings. It is typical in these high-performance applications, however, that the system hardware is simply incapable of saturating the bandwidths supported by the network infrastructure. When the bottleneck for data transfer occurs in the system itself and not in the network, it is critical that the protocol scales gracefully to prevent buffer overflow and packet loss. It is therefore necessary to build a high-speed protocol adaptive to the performance of each system by including a dynamic performance-based flow control. This paper develops such a protocol, performance adaptive UDP (henceforth PA-UDP), which aims to dynamically and autonomously maximize performance under different systems. A mathematical model and related algorithms are proposed to describe the theoretical basis behind effective buffer and CPU management. A novel delay-based rate-throttling model is also demonstrated to be very accurate under diverse system latencies. Based on these models, we implemented a prototype under Linux, and the experimental results demonstrate that PA-UDP outperforms other existing high-speed protocols on commodity hardware in terms of throughput, packet loss, and CPU utilization. PA-UDP is efficient not only for high-speed research networks, but also for reliable high-performance bulk data transfer over dedicated local area networks where congestion and fairness are typically not a concern.