Middleware-based database replication

Emmanuel Cecchet, George Candea, Anastasia Ailamaki
2008 Proceedings of the 2008 ACM SIGMOD international conference on Management of data - SIGMOD '08  
The need for high availability and performance in data management systems has been fueling a long running interest in database replication from both academia and industry. However, academic groups often attack replication problems in isolation, overlooking the need for completeness in their solutions, while commercial teams take a holistic approach that often misses opportunities for fundamental innovation. This has created over time a gap between academic research and industrial practice. This
more » ... paper aims to characterize the gap along three axes: performance, availability, and administration. We build on our own experience developing and deploying replication systems in commercial and academic settings, as well as on a large body of prior related work. We sift through representative examples from the last decade of open-source, academic, and commercial database replication systems and combine this material with case studies from real systems deployed at Fortune 500 customers. We propose two agendas, one for academic research and one for industrial R&D, which we believe can bridge the gap within 5-10 years. This way, we hope to both motivate and help researchers in making the theory and practice of middleware-based database replication more relevant to each other. We find that a few "hot topics" (e.g., reliable multicast and lazy replication [21] ) attract the lion's share of academic interest, while other equally important aspects (e.g., availability and management) are often forgotten-this limits the impact research systems can have on the real world. Motivated by these findings, we draft possible agendas for academic and industrial research. This paper concentrates exclusively on middleware-based 1 replication for OLTP workloads. The prevalent architecture is shared-nothing, where cluster nodes use local disks to store data. We make two contributions. First, we identify gaps between database research and practice on four different levels: RDBMS engine, SQL language, middleware, and system management. We show how overlooking seemingly small details can undermine replication systems. Second, we distill a few research topics that provide low-hanging fruit for closing these gaps, in the realms of middleware design, consistency models, availability, and system evaluation. We also describe what we believe industry ought to do with respect to interfaces, transaction abstractions, system management, and dynamic upgrades. As this paper is not intended to serve as an area survey, the reference list is by no means exhaustive. Rather, we choose representative examples that help us characterize the academia/industry gap and its consequences. The rest of the paper is structured as follows: Section 2 describes the replication schemes presently favored in the field. Section 3 surveys representative academic proposals for replication. Section 4 discusses in detail the practical challenges we have encountered while deploying numerous middleware-based replication systems at customers ranging from small startups to Fortune 500 companies. Section 5 distills the main challenges and outlines roadmaps for both academic research and industrial R&D that can bridge the identified gaps. Section 6 concludes the paper.
doi:10.1145/1376616.1376691 dblp:conf/sigmod/CecchetCA08 fatcat:xpjilgjvs5dihkom7wy5mhwley