Towards understanding modern web traffic

Sunghwan Ihm, Vivek S. Pai
2011 Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference - IMC '11  
As Web sites move from relatively static displays of simple pages to rich media applications with heavy client-side interaction, the nature of the resulting Web traffic changes as well. Understanding this change is necessary in order to improve response time, evaluate caching effectiveness, and design intermediary systems, such as firewalls, security analyzers, and reporting/management systems. Unfortunately, we have little understanding of the underlying nature of today's Web traffic. In this
more » ... aper, we analyze five years (2006-2010) of real Web traffic from a globally-distributed proxy system, which captures the browsing behavior of over 70,000 daily users from 187 countries. Using this data set, we examine major changes in Web traffic characteristics that occurred during this period. We also present a new Web page analysis algorithm that is better suited for modern Web page interactions by grouping requests into streams and exploiting the structure of the pages. Using this algorithm, we analyze various aspects of page-level changes, and characterize modern Web pages. Finally, we investigate the redundancy of this traffic, using both traditional object-level caching as well as content-based approaches.
doi:10.1145/2068816.2068845 dblp:conf/imc/IhmP11 fatcat:uatlh22vx5ac7fz375gftvq5ty