Filters








1,099 Hits in 2.7 sec

Multi-resolution bitmap indexes for scientific data

Rishi Rakesh Sinha, Marianne Winslett
2007 ACM Transactions on Database Systems  
Our experiments with large data sets from two scientific domains show that multi-resolution, parallelizable bitmap indexes occupy an acceptable amount of storage while improving range query performance  ...  In this article, we describe the difficulties we encountered in deploying bitmap indexes with scientific data and queries from two real-world domains.  ...  of Illinois for help with the rocket science data set, and Vikas Mehra and Pratyush Sinha for their help with the earth science data set.  ... 
doi:10.1145/1272743.1272746 fatcat:umx62g5wxbgbrfe6k4zlfbhage

Bitmap indexes for large scientific data sets: a case study

R.R. Sinha, S. Mitra, M. Winslett
2006 Proceedings 20th IEEE International Parallel & Distributed Processing Symposium  
Researchers have previously proposed the use of bitmap indexes for high-dimension scientific data as a way of overcoming the drawbacks of traditional multidimensional indexes such as R-trees and KD-trees  ...  The data used by today's scientific applications are often very high in dimensionality and staggering in size.  ...  We would also like to thank John Norris for his help with the Voyager visualization toolkit.  ... 
doi:10.1109/ipdps.2006.1639304 dblp:conf/ipps/SinhaMW06 fatcat:wf655f4tqvgyfbdssfnvx5mq2e

Effective and efficient data sampling using bitmap indices

Yu Su, Gagan Agrawal, Jonathan Woodring, Kary Myers, Joanne Wendelberger, James Ahrens
2014 Cluster Computing  
We have developed algorithms for using bitmap indices to sample datasets.  ...  We observe that to allow subsetting over scientific datasets, data repositories are likely to use an indexing technique.  ...  Acknowledgments This work was supported by the Department of Energy (DOE) Office of Science (OSC) Advanced Scientific Comput-ing Research (ASCR) and NSF award IIS-0916196 to the Ohio State University.  ... 
doi:10.1007/s10586-014-0360-5 fatcat:etxzxvrqhvedxcti7b7ilqyu7y

NVST data archiving system based on fastbit nosql database [article]

Yingbo Liu, Feng Wang, Kaifan Ji, Hui Deng, Wei Dai, Bo Liang
2016 arXiv   pre-print
Our study brings a new idea for massive astronomical data archiving and would contribute to the design of data management systems for other astronomical telescopes.  ...  In this study, we implement a new data archiving system for the NVST based on the Fastbit Not Only Structured Query Language (NoSQL) database.  ...  Sinha et al. (2006) proposed an adaptive, multi-level and multi-resolution bitmap indexing scheme for scientific data index.  ... 
arXiv:1612.07587v1 fatcat:7yochyaaz5f3jakqiz2m3ig5w4

NVST DATA ARCHIVING SYSTEM BASED ON FASTBIT NOSQL DATABASE

Ying-Bo Liu, Feng Wang, Kai-Fan Ji, Hui Deng, Wei Dai, Bo Liang
2014 Journal of The Korean Astronomical Society  
Our study brings a new idea for massive astronomical data archiving and would contribute to the design of data management systems for other astronomical telescopes.  ...  In this study, we implement a new data archiving system for the NVST based on the Fastbit Not Only Structured Query Language (NoSQL) database.  ...  Sinha et al. (2006) proposed an adaptive, multi-level and multi-resolution bitmap indexing scheme for scientific data index.  ... 
doi:10.5303/jkas.2014.47.3.115 fatcat:wbhmrw5dtrdgnknbpn3ic3otpu

Data Parallel Bin-Based Indexing for Answering Queries on Multi-core Architectures [chapter]

Luke J. Gosink, Kesheng Wu, E. Wes Bethel, John D. Owens, Kenneth I. Joy
2009 Lecture Notes in Computer Science  
This paper presents a new parallel indexing data structure for answering queries that takes full advantage of the increasing thread-level parallelism emerging in multi-core architectures.  ...  The multi-core trend in CPUs and general purpose graphics processing units (GPUs) offers new opportunities for the database community.  ...  for Ultra-Scale Visualization, and the Scientific Data Management Center.  ... 
doi:10.1007/978-3-642-02279-1_9 fatcat:hgzr6wsjwbh25f6tugvy3yrb3y

Parallel index and query for large scale data analysis

Jerry Chou, Rob D. Ryne, Mark Howison, Brian Austin, Kesheng Wu, Ji Qiang, E. Wes Bethel, Arie Shoshani, Oliver Rübel, Prabhat
2011 Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '11  
Modern scientific datasets present numerous data management and analysis challenges.  ...  State-of-the-art index and query technologies are critical for facilitating interactive exploration of large datasets, but numerous challenges remain in terms of designing a system for processing general  ...  For such scientific data sets, the bitmap index is a more appropriate indexing structure [16] [21, Ch. 6].  ... 
doi:10.1145/2063384.2063424 dblp:conf/sc/ChouHAWQBSRPR11 fatcat:ev5ig57zajffbesoq3ucmbuviy

Taming massive distributed datasets

Yu Su, Gagan Agrawal, Jonathan Woodring, Kary Myers, Joanne Wendelberger, James Ahrens
2013 Proceedings of the 22nd international symposium on High-performance parallel and distributed computing - HPDC '13  
We have developed algorithms for using bitmap indices to sample datasets.  ...  We observe that to allow subsetting over scientific datasets, data repositories are likely to use an indexing technique.  ...  Tradeoff between Accuracy and Sampling/Memory Costs: The bitmap indexing allows flexible multi-level indices over a given dataset.  ... 
doi:10.1145/2493123.2462906 fatcat:vhvwifgskvcihgb3ozq3iw6mkq

Secondary indexing in one dimension

Rasmus Pagh, Srinivasa Rao Satti
2009 Proceedings of the twenty-eighth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems - PODS '09  
Secondary indexes are heavily used in relational databases and scientific data analysis.  ...  In this paper we give the first theoretically optimal data structure for the secondary indexing problem.  ...  Using this idea recursively one gets multi-resolution bitmap indexes [16] , illustrated in Figure 1.  ... 
doi:10.1145/1559795.1559824 dblp:conf/pods/PaghS09 fatcat:prx3tgpcfzdh3cv7c5cdbmfh4a

FastQuery: A Parallel Indexing System for Scientific Data

Jerry Chou, Kesheng Wu, Prabhat
2011 2011 IEEE International Conference on Cluster Computing  
However, a challenge is that the indexes assume the relational data model but the scientific data generally follows the array data model.  ...  To match the two data models, we design a generic mapping mechanism and implement an efficient input and output interface for reading and writing the data and their corresponding indexes.  ...  Wes Bethel and Arie Shoshani for their helpful discussions leading up to the design and specification of FastQuery software.  ... 
doi:10.1109/cluster.2011.86 dblp:conf/cluster/ChouWP11 fatcat:oycpgglowzg45awo3isemkiycu

Polygon-Based Query Evaluation over Geospatial Data Using Distributed Hash Tables

Matthew Malensek, Sangmi Pallickara, Shrideep Pallickara
2013 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing  
Data volumes in the geosciences and related domains have grown significantly as sensing equipment designed to continuously gather readings and produce data streams for geographic regions have proliferated  ...  Geoavailability grids are also applicable in non-distributed settings for local lookup functionality, performing competitively with other leading spatial indexing technology.  ...  TABLE I BITMAP I COMPRESSION FOR VARIOUS INDEX RESOLUTIONS Resolution Original Size (KB) Compressed (KB) 15-bit 309.0 294.4 20-bit 9879.02 3196.9 25-bit 316090.28 4034.7 TABLE II INDEX  ... 
doi:10.1109/ucc.2013.46 dblp:conf/ucc/MalensekPP13 fatcat:bngwgw5zmrclzibaa74dj2xvjq

Parallel query evaluation as a Scientific Data Service

Bin Dong, Surendra Byna, Kesheng Wu
2014 2014 IEEE International Conference on Cluster Computing (CLUSTER)  
To answer the queries efficiently, we transparently augment user data with bitmap indexes and ordered datasets.  ...  The generation of bitmap indexes and sorted datasets and querying are parallelized to work on large supercomputers.  ...  data ("using sorted data"), bitmap indexes for the original dataset ("using original data indexes"), and bitmap indexes for the sorted dataset ("using sorted data indexes").  ... 
doi:10.1109/cluster.2014.6968765 dblp:conf/cluster/DongBW14 fatcat:fqiv7tt4uvb23gzu3a3yhnwkly

Decibel

Michael Maddox, David Goehring, Aaron J. Elmore, Samuel Madden, Aditya Parameswaran, Amol Deshpande
2016 Proceedings of the VLDB Endowment  
As scientific endeavors and data analysis become increasingly collaborative, there is a need for data management systems that natively support the versioning or branching of datasets to enable concurrent  ...  analysis, cleaning, integration, manipulation, or curation of data across teams of individuals.  ...  Acknowledgements: This research is funded by NSF under grants 1513972, 1513407, 1513443, and by Intel Science and Technology Center for Big Data.  ... 
doi:10.14778/2947618.2947619 pmid:28149668 pmcid:PMC5278666 fatcat:w7q646klq5ayxfkfd3hzwptz4e

Detecting Distributed Scans Using High-Performance Query-Driven Visualization

Kurt Stockinger, E. Bethel, Scott Campbell, Eli Dart, Kesheng Wu
2006 ACM/IEEE SC 2006 Conference (SC'06)  
Previous Work Our work is based on a multi-disciplinary approach of techniques in network traffic analysis, efficient querying and indexing, and query-driven visualization.  ...  In this paper, we present a set of parallel algorithms that demonstrate how an efficient selection mechanism -bitmap indexing -significantly speeds up a common analysis task, namely, computing conditional  ...  FastBit [Scientific Data Management Group 2006 ] is a research code that implements a number of different forms of bitmap index compression, including WAH.  ... 
doi:10.1109/sc.2006.25 fatcat:6vtrixcekjfbpbq2ubiks6rm5a

Supporting correlation analysis on scientific datasets in parallel and distributed settings

Yu Su, Gagan Agrawal, Jonathan Woodring, Ayan Biswas, Han-Wei Shen
2014 Proceedings of the 23rd international symposium on High-performance parallel and distributed computing - HPDC '14  
Careful analysis of this data holds much promise for future scientific discoveries.  ...  This paper focuses on the problem of correlation analysis across large-scale simulation datasets, including 1) accelerating this analysis with the use of bitmap indexing as a representative summary of  ...  Hierarchical Bitmap Indexing Combining the benefits of both dynamic and static indexing, we have developed a hierarchical bitmap indexing framework to answer correlation analysis for scientific datasets  ... 
doi:10.1145/2600212.2600230 dblp:conf/hpdc/SuAWBS14 fatcat:eg2uk2wx65gizcnhcxqzlz7qzu
« Previous Showing results 1 — 15 out of 1,099 results