WebProfiler: Cooperative diagnosis of Web failures

Sharad Agarwal, Nikitas Liogkas, Prashanth Mohan, Venkata N. Padmanabhan
2010 2010 Second International Conference on COMmunication Systems and NETworks (COMSNETS 2010)  
Despite tremendous growth in the importance and reach of the Web, users have little recourse when a Web page fails to load. Web browsers provide little feedback on such failures, and suggest re-checking the URL or the machine's network settings. Hence, users are often unable to diagnose Web access problems, and resort to haphazardly modifying their settings or simply trying again later. We advocate a client-based collaborative approach for diagnosing Web browsing failures. Our system,
more » ... r, leverages end-host cooperation to pool together observations on the success or failure of Web accesses from multiple vantage points. These are fed into a simple, collaborative blame attribution algorithm. Our evaluation on a controlled testbed shows WebProfiler can accurately diagnose 3.6 times as many failures than possible from a single client's perspective. We present the design and prototype implementation of WebProfiler for an enterprise network. Internet Explorer cannot display the webpage Most likely causes: • You are not connected to the Internet. • The website is encountering problems. • There might be a typing error in the address. -IE-7 error page Despite the tremendous growth of the Web and our reliance on it, a surprisingly large number of attempts to access websites still fail today [18] . The popularity of the Web is fueling an ever increasing number of users which in turn places additional load on HTTP servers and other Web infrastructure. More complex Web applications, such as social networking websites, require server resources beyond that of serving traditional static web pages. The spread of the Internet to various corners of the World is increasing the complexity of Internet infrastructure such as the routing, DNS and CDN systems. Users now expect Web access from more diverse locations, such as mobile phones and the workplace, which typically employ infrastructure such as firewalls and HTTP proxies to filter malicious content, restrict user access and adapt content to different devices. As a result, users do see web failures and they are increasingly difficult to automatically diagnose. When a failure does occur, users are typically presented with vague and generic information about the cause. The less-than-helpful error message from Microsoft Internet Explorer 7 shown above is by no means specific to that browser -Firefox 2's error message is equally useless. Users may try to reload the page repeatedly, modify their machine or browser configuration, or simply try again later. Yet, depending on the cause, none of these actions may resolve the problem, leaving the user clueless and frustrated. There are existing techniques for diagnosing Web failures. HTTP server logs can be analyzed [16] by website administrators to look for requests that generated HTTP errors. However, these logs do not contain access attempts that did not even reach the website (e.g., due to DNS failures, HTTP proxy failures). HTTP proxy or edge router logs can be analyzed by enterprise administrators -medium to large corporations typically use HTTP caching proxies to improve performance and filter harmful web content. These logs include details such as the client IP address, server IP address, URL accessed and HTTP success/error code. However, again, logs collected at the network edge do not include access attempts that failed to reach it (e.g., due to DNS failures, internal routing failures). Recently proposed techniques [9] for constructing inference graphs of multi-level dependencies in enterprise networks use packet traces to probabilistically infer the network components involved in a transaction and identify the culprit if a large number of requests fail. Such systems rely on probabilistic techniques to separate out multiple simultaneous network flows into transactions related to the same application-level action, and are impeded when end-to-end encryption such as IPSec is used. We present WebProfiler, which employs end-host cooperation to pool together observations of Web access failures and successes from multiple vantage points. WebProfiler exploits clients that use different network elements (e.g. DNS servers, HTTP caches) in a collaborative blame attribution algorithm to identify those network elements that are often involved in failures rather than successes. Our distributed system relies on a lightweight Web browser plug-in on each client that monitors the success and failure of any web transaction and shares this with the WebProfiler system. The use of a lightweight application layer plug-in for monitoring results in a system that works even with IPSec encryption, detects all client failures, does not require packet sniffing and can be incrementally deployed via software updates. While the concepts behind WebProfiler are applicable to other networks such as residential access and the Internet in general, we focus on enterprises in our implementation and evaluation. In doing so, we are
doi:10.1109/comsnets.2010.5431995 dblp:conf/comsnets/AgarwalLMP10 fatcat:u7kw4iq525hf7jwkom32djs6f4