INT-filter: Mitigating Data Collection Overhead for High-Resolution In-band Network Telemetry

• Published in: GLOBECOM 2020 - 2020 IEEE Global Communications Conference pp. 1 - 6
• Main Authors: Song, Enge, Pan, Tian, Jia, Chenhao, Cao, Wendi, Zhang, Jiao, Huang, Tao, Liu, Yunjie
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2020
• Subjects: Bandwidth; Data collection; Probes; Redundancy; Telemetry; Time-frequency analysis; Topology
• ISSN: 2576-6813
• DOI: 10.1109/GLOBECOM42002.2020.9348029

In-band Network Telemetry (INT) enables fine-grained network monitoring to ease the management of large-scale networks, which, however, relies on the real-time collection of a huge amount of telemetry data through the southbound interface. For example, the INT telemetry data upload rate of a 28-pod FatTree topology reaches 3Tbps under a probe frequency of 100 times/s, which is rather unacceptable since the controller-switch link bandwidth is limited. To mitigate the telemetry data collection overhead, in this work, we propose INT-filter, a novel measurement architecture that deploys the same prediction algorithm on both the data plane and the control plane to predict the traffic state in the near future instead of uploading all the telemetry data. Such prediction-based approach leverages the observation that there is considerable redundancy in the telemetry data sequence. In addition, we design an integration mechanism that conducts predictions using multiple methods simultaneously and uploads the predicted result from the least-error method to further decrease the upload volume. Extensive evaluation suggests that INT-filter can achieve at least 33.6% data collection decrease under a 10ms probe interval. With prediction integration, the upload reduction can further reach 58.5%.