Fusing effectful comprehensions

Olli Saarikivi, Margus Veanes, Todd Mytkowicz, Madan Musuvathi
<span title="">2017</span> <i title="ACM Press"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/jpubwsjaozha5itdes6pzyz2fm" style="color: black;">Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation - PLDI 2017</a> </i> &nbsp;
List comprehensions provide a powerful abstraction mechanism for expressing computations over ordered collections of data declaratively without having to use explicit iteration constructs. This paper puts forth effectful comprehensions as an elegant way to describe list comprehensions that incorporate loop carried state. This is motivated by operations such as compression/decompression and serialization/deserialization that are common in log/data processing pipelines and require loop-carried
more &raquo; ... te when processing an input stream of data. We build on the underlying theory of symbolic transducers to fuse pipelines of effectful comprehensions into a single representation, from which efficient code can be generated. Using background theory reasoning with an SMT solver our fusion and subsequent reachability based branch elimination algorithms can significantly reduce the complexity of the fused pipelines. Our implementation shows significant speedups over reasonable hand-written code (3×, on average) and a LINQ implementation of the pipeline (5×, on average) for a variety of examples, including scenarios for extracting fields with regular expressions, processing XML with XPath, and running queries over encoded data. Finally, we formalize the semantics of symbolic transducers and their compositions as a transduction monad, which provides a link between the automata-theoretic view and a monadic view of symbolic transducers.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/3062341.3062362">doi:10.1145/3062341.3062362</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/pldi/SaarikiviVMM17.html">dblp:conf/pldi/SaarikiviVMM17</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/567q5l43zrcwjpwwaftuvhqsti">fatcat:567q5l43zrcwjpwwaftuvhqsti</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20161215230351/https://www.microsoft.com/en-us/research/wp-content/uploads/2016/10/fusing_effectful_comprehensions.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/85/ef/85ef581d482f66795d875d0712ba0084928a6f82.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/3062341.3062362"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> acm.org </button> </a>