Anemone

Michael R. Hines, Mark Lewandowski, Kartik Gopalan
2005 Proceedings of the twentieth ACM symposium on Operating systems principles - SOSP '05  
I. MOTIVATION AND PROBLEM STATEMENT Large-memory high-performance applications such as scientific computing, weather prediction simulations, database management, and graphics rendering applications are increasingly claiming a prominent share of the cluster-based application space. Even though improvements in Dynamic Random Access Memory (DRAM) technology have yielded an enormous increase in the memory capacity of off-theshelf systems, the memory-intensive applications continue to hunger for
more » ... larger amounts of memory. The issue is not whether one can provide enough DRAM to satisfy these applications; rather the more DRAM one provides, the more these applications ask for. These large-memory applications quickly hit the physical memory limit and start paging to the local disk. As a consequence, application execution time is degraded due to higher disk latency involved in paging operations. At the same time, while memory resources in one machine in a LAN-based cluster might be heavily loaded, large amounts of memory in other machines might remain idle or under-utilized. Since affordable Gigabit LANs with attractive cost-to-performance ratios are now very common, one could significantly reduce access latencies by first paging over a high-speed LAN to the unused memory of remote machines and then resort to disk-based paging only as the last resort when remote resources are full. Thus, remote memory access can be viewed as another level between the traditional memory hierarchy memory and the disk. In fact, typical remote memory paging latencies of 100 to 500µs can be easily achieved whereas the latency of paging to slow local disk (especially while paging in) is typically around 2 to 10ms, depending upon seek and rotational overheads. Thus remote memory paging could potentially be one to two orders of magnitude faster than paging to disk [13] . II. RESEARCH OBJECTIVES Our primary research objective, called the Anemone (Adaptive NEtwork MemOry engiNE) project, is to harness a large collective pool of memory resources across nodes in a high-speed LAN. The ultimate performance objective of Anemone is to enable memory hungry applications to gain transparent, fault-tolerant, and low-overhead access to potentially "unlimited" memory resources. The novel aspect that distinguishes the Anemone project from prior efforts is that no modifications are required either to the memoryintensive applications or to the end-hosts, thus eliminating painstaking and expensive end-system modifications. Instead, Anemone exploits the standard features that arrive bundled with these systems -the RAM disk (or memory-resident disk interface) and the Network File System (NFS) protocol -in a novel manner to provide transparent and low-latency access to remote-memory. Virtualization is emerging as a fundamental paradigm that will drive the provisioning and management of resources in future large scale clusters. Virtualization is the ability to subdivide a pool of physical resources, such as computation, memory, storage, and bandwidth, into higher level logical partitions, each of which can be assigned to individual users of the resource pool. This high-level view of a cluster's resources greatly simplifies their provisioning and management. In this work, we specifically tackle the challenge of virtualizing the distributed memory resources in a cluster to support the memory needs of high-performance applications. III. STATE OF THE ART While the advantages of accessing memory over the network have been explored in prior works, their acceptance by high performance application community in running realworld memory-intensive applications has been scant in spite of their enormous potential benefits. The primary reason is that all earlier approaches advocate significant modifications to the end-systems that need the extra memory. These modifications range from application-specific programming interfaces that programmers must follow to changes in the end-host's operating system (OS) and device drivers. In early 1990's the first two remote paging mechanisms were proposed [4], [8], both of which employed a customized remote paging protocol between clients and servers and incorporated extensive OS changes. Global Memory System (GMS) [7] provides network-wide memory management support at the lowest level of the OS for activities such as paging, memory mapped files and file caching. The Network RamDisk [9] offers remote paging with data replication and adaptive parity caching. Network RamDisk is implemented as a device driver inside the OS, thus increasing the installation and management complexity at each node in the LAN. Dodo [10] provides a user-level library based interface that a programmer that coordinates all data transfer to and from a remote memory cache. Samson [12] is a dedicated memory
doi:10.1145/1095810.1118608 fatcat:nbqkjksxpffdflehshtz2s6gom