Filters








13 Hits in 3.8 sec

A Bulk-Parallel Priority Queue in External Memory with STXXL [chapter]

Timo Bingmann, Thomas Keh, Peter Sanders
2015 Lecture Notes in Computer Science  
We propose the design and an implementation of a bulkparallel external memory priority queue to take advantage of both sharedmemory parallelism and high external memory transfer speeds to parallel disks  ...  Our experimental results show that in the selected benchmarks the priority queue reaches 64% of the full parallel I/O bandwidth of SSDs and 49% of rotational disks, or the speed of sorting in external  ...  This is because the EM containers in STXXL follow the C++ STL's interface and std::priority queue is a max-heap. Other PQ Implementations.  ... 
doi:10.1007/978-3-319-20086-6_3 fatcat:4vlhjbcisnhsfgvyolqitgau2i

STXXL: standard template library for XXL data sets

R. Dementiev, L. Kettner, P. Sanders
2008 Software, Practice & Experience  
STXXL is the only external memory algorithm library supporting parallel disks.  ...  The STL-user layer provides external memory sorting, external memory stack, external memory priority queue, etc. which have (almost) the same interfaces (including syntax and semantics) as their STL counterparts  ...  We are also working on parallelizing external priority queues and on using task-based parallelism to implement pipelining.  ... 
doi:10.1002/spe.844 fatcat:j3ngetudlvacxdwsa6uldcidra

Stxxl: Standard Template Library for XXL Data Sets [chapter]

Roman Dementiev, Lutz Kettner, Peter Sanders
2005 Lecture Notes in Computer Science  
STXXL is the only external memory algorithm library supporting parallel disks.  ...  The STL-user layer provides external memory sorting, external memory stack, external memory priority queue, etc. which have (almost) the same interfaces (including syntax and semantics) as their STL counterparts  ...  We are also working on parallelizing external priority queues and on using task-based parallelism to implement pipelining.  ... 
doi:10.1007/11561071_57 fatcat:mc4vj65kjfe7jgt52w4goal2z4

Design and Implementation of a Practical I/O-efficient Shortest Paths Algorithm [chapter]

Ulrich Meyer, Vitaly Osipov
2009 2009 Proceedings of the Eleventh Workshop on Algorithm Engineering and Experiments (ALENEX)  
Furthermore, compared to the currently best implementation for external-memory BFS [6] , which in a sense constitutes a lower bound for SSSP, the running time of our approach always stayed within a factor  ...  Finally, we also report on early results of experiments in which we replace the hard disk by a solid state disk (flash memory).  ...  We would like to thank Deepak Ajwani and Andreas Beckmann for helpful discussions and assistance with the flash disks.  ... 
doi:10.1137/1.9781611972894.9 dblp:conf/alenex/MeyerO09 fatcat:occf2vj5ynawvkfdgaep7m26vu

External memory K-bisimulation reduction of big graphs

Yongming Luo, George H.L. Fletcher, Jan Hidders, Yuqing Wu, Paul De Bra
2013 Proceedings of the 22nd ACM international conference on Conference on information & knowledge management - CIKM '13  
in external memory.  ...  Ubiquitous in the theory and application of graph data, bisimulation is a robust notion of node equivalence which intuitively groups together nodes in a graph which share fundamental structural features  ...  We use a priority queue pQueue to record and process such changes in a systematic, level-wise manner. For some node ID uId and iteration j, pQueue stores the pair (j,uId) as priority reference.  ... 
doi:10.1145/2505515.2505752 dblp:conf/cikm/LuoFHWB13 fatcat:iyu5xa5hdnfsvoimqytzygtdwy

Better external memory suffix array construction

Roman Dementiev, Juha Kärkkäinen, Jens Mehnert, Peter Sanders
2008 ACM Journal of Experimental Algorithmics  
Suffix arrays are a simple and powerful data structure for text processing that can be used for full text indexes, data compression, and many other applications in particular in bioinformatics.  ...  The algorithms are asymptotically optimal in the worst case or on the average. Our implementation can construct suffix arrays for inputs of up to 4GBytes in hours on a low cost machine.  ...  Lutz Kettner helped with the design of Stxxl. The html pages were supplied by Sergey Sizov from the information retrieval group at MPII.  ... 
doi:10.1145/1227161.1402296 fatcat:fyyvsjdpbzemraaopswnrlh5va

Chapter 5. Realistic Computer Models [chapter]

Deepak Ajwani, Henning Meyerhenke
2010 Lecture Notes in Computer Science  
Many variants of external priority queue also do lazy processing of decrease-key operations after collecting them in a batch.  ...  Many priority queue applications in graph algorithms are examples of this.  ... 
doi:10.1007/978-3-642-14866-8_5 fatcat:j326q2ymeffzfmo36nqst7msmq

Bulk-Parallel Priority Queue in External Memory

Thomas Keh
2014
Juli 2014 iii Abstract We present a priority queue implementation with support for external memory. The focus of our work has been to derive a benefit from parallel sharedmemory machines.  ...  It's the first parallel optimization of an external-memory priority queue.  ...  Bulk-Parallel Priority Queue in External Memory This chapter presents the bulk-parallel priority queue that has originated from this work.  ... 
doi:10.5445/ir/1000042468 fatcat:3vdfceyom5ehdimrhcgr7q4mge

Algorithms and Data Structures for External Memory

Jeffrey Scott Vitter
2006 Foundations and Trends® in Theoretical Computer Science  
A variety of EM paradigms are considered for solving batched and online problems efficiently in external memory.  ...  The resulting input/output communication (or I/O) between fast internal memory and slower external memory (such as disks) can be a major performance bottleneck.  ...  STXXL also supports a library of standard data structures, such as stacks, queues, search trees, and priority queues.  ... 
doi:10.1561/0400000014 fatcat:bkfchugd4fbjvcu5zipnh23k6e

Algorithm Libraries for Multi-Core Processors

Johannes Singler
2010
Some parallelized geometric algorithms are introduced into CGAL. Further, we design and implement sorting algorithms for huge data in distributed external memory.  ...  The Multi-Core STL provides basic algorithms for internal memory, while the parallelized STXXL enables multi-core acceleration for algorithms on large data sets stored on disk.  ...  The STXXL provides several external memory data structures, the priority queue being the only one that is compute-intensive.  ... 
doi:10.5445/ir/1000019394 fatcat:7ysnoovp3je3rn4ytnitqw4j6q

Traversing large graphs in realistic settings [article]

Deepak Ajwani, Universität Des Saarlandes, Universität Des Saarlandes
2009
external memory model do not realize the full potential of the flash memory devices.  ...  We present a simple algorithm which maintains the topological order of a directed acyclic graph with n nodes under an online edge insertion sequence in O(n 2.75 ) time, independent of the number m of edges  ...  She had been encouraging and understanding, even as I had been mentally immersed in my work.  ... 
doi:10.22028/d291-25941 fatcat:zt6luwhokvh7npqcwoidyd2dty

Algorithm Engineering for fundamental Sorting and Graph Problems

Vitaly Osipov
2014
memory or available parallelism.  ...  It is a set of basic techniques one can find in any (good) coursebook on algorithms and data structures.  ...  Also thanks are due to Dominik Schultes and Roman Dementiev for their help in using the external MST implementation and STXXL, respectively.  ... 
doi:10.5445/ir/1000042377 fatcat:72infiexarhixk6vxfg6b7op4i

Scalable String and Suffix Sorting: Algorithms, Techniques, and Tools [article]

Timo Bingmann
2018 pre-print
external memory suffix sorting with a new distributed algorithmic big data framework named Thrill.  ...  The first part considers parallel string sorting on shared-memory multi-core machines, the second part external memory suffix sorting using the induced sorting principle, and the third part distributed  ...  Acknowledgment We would like thank the AWS Cloud Credits for Research program for making the experiments in section 7.4 possible.  ... 
doi:10.5445/ir/1000085031 arXiv:1808.00963v1 fatcat:dwu6krhxfnc4bmjprangajvvpe