Filters








4 Hits in 0.67 sec

Samza

Shadi A. Noghabi, Kartik Paramasivam, Yi Pan, Navina Ramesh, Jon Bringhurst, Indranil Gupta, Roy H. Campbell
<span title="2017-08-01">2017</span> <i title="VLDB Endowment"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/p6rqwwpkkjbcldejepcehaalby" style="color: black;">Proceedings of the VLDB Endowment</a> </i> &nbsp;
Distributed stream processing systems need to support stateful processing, recover quickly from failures to resume such processing, and reprocess an entire data stream quickly. We present Apache Samza, a distributed system for stateful and fault-tolerant stream processing. Samza utilizes a partitioned local state along with a low-overhead background changelog mechanism, allowing it to scale to massive state sizes (hundreds of TB) per application. Recovery from failures is sped up by
more &raquo; ... g based on Host Affinity. In addition to processing infinite streams of events, Samza supports processing a finite dataset as a stream, from either a streaming source (e.g., Kafka), a database snapshot (e.g., Databus), or a file system (e.g. HDFS), without having to change the application code (unlike the popular Lambdabased architectures which necessitate maintenance of separate code bases for batch and stream path processing). Samza is currently in use at LinkedIn by hundreds of production applications with more than 10, 000 containers. Samza is an open-source Apache project adopted by many top-tier companies (e.g., LinkedIn, Uber, Netflix, TripAdvisor, etc.). Our experiments show that Samza: a) handles state efficiently, improving latency and throughput by more than 100× compared to using a remote storage; b) provides recovery time independent of state size; c) scales performance linearly with number of containers; and d) supports reprocessing of the data stream quickly and with minimal interference on real-time traffic.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.14778/3137765.3137770">doi:10.14778/3137765.3137770</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ameij3w5m5a6bcq62a2k5mwolm">fatcat:ameij3w5m5a6bcq62a2k5mwolm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20180723143512/http://www.vldb.org/pvldb/vol10/p1634-noghabi.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/97/5a/975a9eb18929faf31c653d3d6d923fc07c3806d2.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.14778/3137765.3137770"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Building a replicated logging system with Apache Kafka

Guozhang Wang, Joel Koshy, Sriram Subramanian, Kartik Paramasivam, Mammad Zadeh, Neha Narkhede, Jun Rao, Jay Kreps, Joe Stein
<span title="2015-08-01">2015</span> <i title="VLDB Endowment"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/p6rqwwpkkjbcldejepcehaalby" style="color: black;">Proceedings of the VLDB Endowment</a> </i> &nbsp;
Apache Kafka is a scalable publish-subscribe messaging system with its core architecture as a distributed commit log. It was originally built at LinkedIn as its centralized event pipelining platform for online data integration tasks. Over the past years developing and operating Kafka, we extend its log-structured architecture as a replicated logging backbone for much wider application scopes in the distributed environment. In this abstract, we will talk about our design and engineering
more &raquo; ... e to replicate Kafka logs for various distributed data-driven systems at LinkedIn, including source-of-truth data storage and stream processing.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.14778/2824032.2824063">doi:10.14778/2824032.2824063</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/e3ul3mxtcjeyjjikhix4mvo4bq">fatcat:e3ul3mxtcjeyjjikhix4mvo4bq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20151031193659/http://www.vldb.org/pvldb/vol8/p1654-wang.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/7e/02/7e02c7f9081f053e68324d6f021aba4f2d09fc21.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.14778/2824032.2824063"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Railgun: managing large streaming windows under MAD requirements [article]

Ana Sofia Gomes, João Oliveirinha, Pedro Cardoso, Pedro Bizarro
<span title="2021-06-23">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Noghabi, Kartik Paramasivam, Yi Pan, Navina Ramesh, Jon Bringhurst, faro, and Alfredo Cuzzocrea (Eds.). IEEE Computer Society, 143–152. https: Indranil Gupta, and Roy H.  ...  Noghabi, Kartik Paramasivam, Yi Pan, Navina Ramesh, Jon Bringhurst, the 2000 ACM SIGMOD International Conference on Management of Data, May Indranil Gupta, and Roy H.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2106.12626v1">arXiv:2106.12626v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/otrohxszszay7dwp4cdmcrfqke">fatcat:otrohxszszay7dwp4cdmcrfqke</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210626123343/https://arxiv.org/pdf/2106.12626v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/b4/e9/b4e9c0e9bbba7fe17ff8170a50a500af2db475eb.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2106.12626v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Presentation Schedule

<span title="2020-11-06">2020</span> <i title="IEEE"> 2020 IEEE International Conference for Innovation in Technology (INOCON) </i> &nbsp;
Paramasivam Design of Low complexity&High performance LUT based FeedForward FFT architecture 10.16 AM 10.30 AM 8 326 Ronak Sarikhada and Prashant K Shah SPEED UP THE VALIDATION PROCESS BY  ...  Real Time Skeletal Hand Muscle Strength 9.31 AM 9.45 AM 5 183 Bhanupriya Mishra and Neelam Sobha Nirala A survey on denoising techniques of PPG signal 9.46 AM 10. 00 AM 6 307 Ria Rawal, Kartik  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/inocon50539.2020.9298402">doi:10.1109/inocon50539.2020.9298402</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7gj4wwbskjedlbsl7csho3pcf4">fatcat:7gj4wwbskjedlbsl7csho3pcf4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210102032030/https://ieeexplore.ieee.org/ielx7/9297926/9298025/09298402.pdf?tp=&amp;arnumber=9298402&amp;isnumber=9298025&amp;ref=" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/0e/00/0e007f058203caaa4e5dbbcfdf4dd66aa505f99d.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/inocon50539.2020.9298402"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>