Filters








2,669 Hits in 10.7 sec

Translation of Array-Based Loops to Distributed Data-Parallel Programs [article]

Leonidas Fegaras, Md Hasanuzzaman Noor
2020 arXiv   pre-print
We present a novel framework for translating programs expressed as array-based loops to distributed data parallel programs that is more general and efficient than related work.  ...  Scientists, who are typically comfortable with numerical analysis tools but are not familiar with the intricacies of Big Data analytics, must now learn to convert their loop-based programs to distributed  ...  and translates the loop-based program Translation of Array-Based Loops to Distributed Data-Parallel Programs where i 1 1 , . . . , i k are loop indexes and c 0 , . . . , c k are constants.  ... 
arXiv:2003.09769v1 fatcat:enwokbtxrravrj32szinem7kc4

Translation of array-based loops to distributed data-parallel programs

Leonidas Fegaras, Hasanuzzaman Noor
2020 Proceedings of the VLDB Endowment  
We present a novel framework for translating programs expressed as array-based loops to distributed data parallel programs that is more general and efficient than related work.  ...  Scientists, who are typically comfortable with numerical analysis tools but are not familiar with the intricacies of Big Data analytics, must now learn to convert their loop-based programs to distributed  ...  CONCLUSION We have addressed the problem of automated parallelization of array-based loops by translating them to comprehensions, which can then be translated and optimized to distributed data parallel  ... 
doi:10.14778/3389133.3389141 fatcat:xp2vaebvibhrfckxgzbsd2kg3y

HiFrames: High Performance Data Frames in a Scripting Language [article]

Ehsan Totoni, Wajih Ul Hassan, Todd A. Anderson, Tatiana Shpeisman
2017 arXiv   pre-print
We demonstrate that HiFrames is significantly faster than alternatives such as Spark SQL on clusters, without forcing the programmer to switch to embedded SQL for part of the program.  ...  HiFrames then automatically parallelizes and compiles relational operations along with other array computations in end-to-end data analytics programs, and generates efficient MPI/C++ code.  ...  It compiles and optimizes SQL to Java byte code that runs on top of RDD APIs for distributed execution.  ... 
arXiv:1704.02341v1 fatcat:so22chmjfncs5dtg3mgisiahwy

An Extended Form of MATLAB To-map Reduce Frameworks in HADOOP Based Cloud Computing Environments

T. Tamilvizhi, B. Parvatha Varthini, K. Manoj, R. Surendran
2016 Research Journal of Applied Sciences Engineering and Technology  
Aim of study to extend the implementation of Matlab to Mapreduce translation based on the M2M translation technique.  ...  Thus to achieve this parallelism Hadoop Distributed File System (HDFS) uses Mapreduce as a programming index.  ...  ACKNOWLEDGMENT The authors would like to thank Sathyabama Univesrsity (India) and Special thanks go to Sur University College (Sultanate of Oman) for providing us with various resources and an unconditional  ... 
doi:10.19026/rjaset.12.2807 fatcat:smzzrzrmgjetjakj4vsvo4dofi

The BUDS Language for Distributed Bayesian Machine Learning

Zekai J. Gao, Shangyu Luo, Luis L. Perez, Chris Jermaine
2017 Proceedings of the 2017 ACM International Conference on Management of Data - SIGMOD '17  
All of this allows for the BUDS compiler to automatically perform deep optimizations of the user's program, and automatically generate efficient implementations.  ...  The types supported in BUDS-vectors, arrays, etc.-are simply logical abstractions useful for programming, and do not correspond to the actual implementation.  ...  The SQL optimizer is then used as a black box to search among different relational algebra implementations for the resulting SQL program.  ... 
doi:10.1145/3035918.3035937 dblp:conf/sigmod/GaoLPJ17 fatcat:if3w3wn6wvfwna4sbapvzei7qu

Declarative Data Analytics: a Survey [article]

Nantia Makrynioti Athens University of Economics, Business)
2019 arXiv   pre-print
The survey explores a wide range of declarative data analysis frameworks by examining both the programming model and the optimization techniques used, in order to provide conclusions on the current state  ...  It proposes declarative languages for expressing data analysis tasks and develops systems which optimize programs written in those languages.  ...  ACKNOWLEDGMENTS We thank Panagiotis-Ioannis Betchavas for the implementation of Linear Regression using DML in section 4.4.1.  ... 
arXiv:1902.01304v1 fatcat:mixepfprkjc5xayhz76bwu3px4

PQL: A Purely-Declarative Java Extension for Parallel Programming [chapter]

Christoph Reichenbach, Yannis Smaragdakis, Neil Immerman
2012 Lecture Notes in Computer Science  
We present an approach where parallel programming takes place in a restricted (sub-Turing-complete), logic-based declarative language, embedded in Java.  ...  This approach offers a key property: the purely declarative nature of our language allows for aggressive optimization, in much the same way that relational queries are optimized by a database engine.  ...  We would like to thank the anonymous ECOOP reviewers for their feedback. This work was funded by the National Science Foundation under grants CCF-0917774, CCF-0934631, and CCF-1115448.  ... 
doi:10.1007/978-3-642-31057-7_4 fatcat:oiauxv2vfverbpfq5qxclzbrzq

Extracting Equivalent SQL from Imperative Code in Database Applications

K. Venkatesh Emani, Karthik Ramachandra, Subhro Bhattacharya, S. Sudarshan
2016 Proceedings of the 2016 International Conference on Management of Data - SIGMOD '16  
In this paper we present an approach to this problem which is based on extracting a concise algebraic representation of (parts of) an application, which may include imperative code as well as SQL queries  ...  The algebraic representation can then be translated into SQL to improve application performance, by reducing the volume of data transferred, as well as reducing latency by minimizing the number of network  ...  The work of Karthik Ramachandra was supported by a fellowship from Microsoft Research, India.  ... 
doi:10.1145/2882903.2882926 dblp:conf/sigmod/EmaniRBS16 fatcat:hj2adxwksrbjjl5xkmgrakfkkm

How to Architect a Query Compiler

Amir Shaikhha, Yannis Klonatos, Lionel Parreaux, Lewis Brown, Mohammad Dashti, Christoph Koch
2016 Proceedings of the 2016 International Conference on Management of Data - SIGMOD '16  
and sustainable compiler-based data management systems.  ...  We propose to use a stack of multiple DSLs on different levels of abstraction with lowering in multiple steps to make query compilers easier to build and extend, ultimately allowing us to create more convincing  ...  Acknowledgments We thank Vojin Jovanovic and Manohar Jonalagedda for insightful discussions that helped to improve this paper. This work was supported by NCCR MARVEL and ERC grant 279804.  ... 
doi:10.1145/2882903.2915244 dblp:conf/sigmod/ShaikhhaKPBD016 fatcat:tlwazhetpna4pmmel7iscxmqfm

Data set preprocessing and transformation in a database system

Carlos Ordonez
2011 Intelligent Data Analysis  
We then provide specific guidelines to translate programs written in a traditional programming language into SQL statements.  ...  Based on successful real-life projects, we present time performance comparisons between SQL code running inside the database system and external data mining programs.  ...  Rote from Teradata Corporation for his valuable guidance and support to migrate many data mining projects into the DBMS.  ... 
doi:10.3233/ida-2011-0485 fatcat:7jifmmcbdbbmribvdylnx3eqri

Optimizing database-backed applications with query synthesis

Alvin Cheung, Armando Solar-Lezama, Samuel Madden
2013 Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation - PLDI '13  
The theory is close in expressiveness to SQL, so the synthesized postconditions can be readily translated to SQL queries.  ...  Such application code does not take advantage of the optimized relational implementations that database systems provide, such as efficient implementations of joins or push down of selection predicates.  ...  We demonstrate a new approach to compiler optimization based on constraint-based synthesis of loop invariants and apply it to the problem of transforming low-level loop nests into high-level SQL queries  ... 
doi:10.1145/2491956.2462180 dblp:conf/pldi/CheungSM13 fatcat:cdjekm7n75ht7cmikqut3aihgq

Weld: Rethinking the Interface Between Data-Intensive Applications [article]

Shoumik Palkar, James Thomas, Deepak Narayanan, Anil Shanbhag, Rahul Palamuttam, Holger Pirk, Malte Schwarzkopf, Saman Amarasinghe, Samuel Madden,, Matei Zaharia
2017 arXiv   pre-print
Even when each function is optimized in isolation, the performance of the combined application can be an order of magnitude below hardware limits due to extensive data movement across these functions.  ...  To address this problem, we propose Weld, a new interface between data-intensive libraries that can optimize across disjoint libraries and functions.  ...  This research was supported in part by affiliate members and other supporters of the Stanford DAWN  ... 
arXiv:1709.06416v2 fatcat:nda4d32uafctpcql743lx6qofy

Optimizing database-backed applications with query synthesis

Alvin Cheung, Armando Solar-Lezama, Samuel Madden
2013 SIGPLAN notices  
The theory is close in expressiveness to SQL, so the synthesized postconditions can be readily translated to SQL queries.  ...  Such application code does not take advantage of the optimized relational implementations that database systems provide, such as efficient implementations of joins or push down of selection predicates.  ...  We demonstrate a new approach to compiler optimization based on constraint-based synthesis of loop invariants and apply it to the problem of transforming low-level loop nests into high-level SQL queries  ... 
doi:10.1145/2499370.2462180 fatcat:q67ysla4ybc3jnrhx4jgdespkq

Steno

Derek Gordon Murray, Michael Isard, Yuan Yu
2011 Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation - PLDI '11  
Steno translates a declarative LINQ query into type-specialized, inlined and loop-based imperative code. It eliminates chains of iterators from query execution, and optimizes nested queries.  ...  By increasing the level of abstraction over imperative code, they improve program readability and, crucially, create opportunities for automatic parallelization and optimization.  ...  We would also like to thank Steve Hand and the anonymous reviewers, whose comments on earlier drafts of this paper have been invaluable for improving the presentation of this work.  ... 
doi:10.1145/1993498.1993513 dblp:conf/pldi/MurrayIY11 fatcat:t4a2zw6j3rfwljjrxmmsp2tkxa

Steno

Derek Gordon Murray, Michael Isard, Yuan Yu
2012 SIGPLAN notices  
Steno translates a declarative LINQ query into type-specialized, inlined and loop-based imperative code. It eliminates chains of iterators from query execution, and optimizes nested queries.  ...  By increasing the level of abstraction over imperative code, they improve program readability and, crucially, create opportunities for automatic parallelization and optimization.  ...  We would also like to thank Steve Hand and the anonymous reviewers, whose comments on earlier drafts of this paper have been invaluable for improving the presentation of this work.  ... 
doi:10.1145/2345156.1993513 fatcat:lzoqp4hlhnbkjkqw4vft7llw2m
« Previous Showing results 1 — 15 out of 2,669 results