We present the Measurement Manager Protocol (MGRP), an in-kernel service that schedules and transmits probes on behalf of active measurement tools. Unlike prior measurement services, MGRP transparently piggybacks application packets inside the often significant amounts of empty padding contained in typical probes. Using MGRP thus combines the modularity, flexibility, and accuracy of standalone active measurement tools with the lower overhead of passive measurement techniques. Microbenchmark

more »

... riments show that the resulting bandwidth savings makes it possible to measure the network accurately, but faster and more aggressively than without piggybacking, and with few ill effects to piggybacked application or competing traffic. When using MGRP to schedule measurements on behalf of MediaNet, an overlay service that adaptively schedules media streams, we show MediaNet can achieve significantly higher streaming rates under the same network conditions. This paper presents the Measurement Manager Protocol (MGRP), an in-kernel service for performing network measurement that aims to address these shortcomings. The main novelty of MGRP is that it permits measurement algorithms to be written as if active, but implemented as if passive; in doing so, it enables measurement algorithms to be more aggressive in their approach, without harming application performance. MGRP works by transparently piggybacking application data inside probes, which mostly consist of empty padding. Rather than send their probe packets directly (e.g., via UDP), active measurement tools use the MGRP Probe API to specify an entire train of probes (by specifying the number of probes, the size of each probe, the amount of padding, and the gap between probes). MGRP treats each probe as a vessel that can potentially carry useful payload; data packets bound for the same destination ride along in the probe and are extracted and delivered separately when the probes arrive. By filling most or all probe padding with useful data, MGRP allows active algorithms to approach the low overhead of passive algorithms, while retaining the accuracy, flexibility, and broader applicability of active probing. Measurement can be performed more aggressively, without worry of negatively impacting application traffic. Indeed, MGRP addresses our motivating example just as effectively as customized approaches [16] that are application-specific, but in a more modular, reusable fashion, which scales naturally to more applications and measurement algorithms. For example, we have implemented a measurement overlay network that actively measures its virtual links; applications query the overlay to acquire up-to-date path conditions. By running over MGRP, overlay probes can piggyback any application traffic traversing the same virtual links, and any active algorithm added to the overlay will immediately receive the benefits of piggybacking. We have written a loadable kernel module for the Linux 2.6.25 kernel that implements MGRP as a Layer 4 transport protocol, and modified TCP to contribute packets to it (Section 2). 1 We have also modified three measurement tools to use the Probe API, including pathload [14] and pathchirp [23] for available bandwidth estimation, and badabing [28] for loss estimation. We have implemented a measurement overlay network to use MGRP-aware pathload to measure available bandwidth, and show how a modified version of the MediaNet streaming media overlay network [11] solves the upscaling problem by using the overlay measurements (Section 3). Microbenchmarks and experiments with MediaNet (Sections 4 and 5) show that MGRP is efficient, fair, and timely-MGRP improves the quality of service available to application source traffic and cross traffic, and enables measurement algorithms to probe more aggressively and thus complete more quickly. MGRP is available for download at