Improving main memory utilization for array-based datacube computation

Seigo Muto, Masaru Kitsuregawa
1998 Proceedings of the 1st ACM international workshop on Data warehousing and OLAP - DOLAP '98  
Computing datacubes requires multidimensional aggregations for all possible combinations of each dimension. In thii paper, we present a method to improve main memory utilization efficiency for an array-based algorithm for datacube computation in a MOLAP context. The problem with the array-based algorithm is in its sparsity, where a large pre portion of array cells are empty. ,The algorithm proposed in [ZDN97] reduces this space inefficiency by compressing arrays on disk. We improve on this
more » ... ithm by performing compression of arrays in main memory as well as on disk using a hashing method, which allocates main memory according to the number of non-empty array cells. We further improve the algorithm using a dynamic main memory allocation strategy. The algorithm by [ZDN97] computes the multiple aggregate views simultaneously, which consumes a lot of main memory space. We propose a main memory allocation method that minimizes the main memory requirement by dynamically allocating main memory only to necessary aggregate views at run time. These savings in main memory resources result in the reduction of disk I/O cost. We evaluate the performance of the proposed method by disk I/O analysis and demonstrate that the improved MOLAP algorithm compares well with a ROLAP algorithm.
doi:10.1145/294260.294267 dblp:conf/dolap/MutoK98 fatcat:5kzbid7hnzeh5mszukgbbcdcly