Compact Data Structures for Network Telemetry

Collecting and analyzing of network traffic data (network telemetry) plays a critical role in managing modern networks. Network administrators analyze their traffic to troubleshoot performance and reliability problems and to detect and block cyberattacks. However, conventional traffic-measurement techniques offer limited visibility into network conditions and rely on offline analysis. Fortunately, network devices-such as switches and network interface cards-are increasingly programmable at the packet level, enabling flexible analysis of the traffic in place, as the packets fly by. However, to operate at high speed, these devices have limited memory and computational resources, leading to trade-offs between accuracy and overhead. In response, an exciting research area emerged, bringing ideas from compact data structures and streaming algorithms to bear on important networking telemetry applications and the unique characteristics of high-speed network devices. In this article, we review the research on compact data structures for network telemetry and discuss promising directions for future research.