Managing scientific data

Anastasia Ailamaki, Verena Kantere, Debabrata Dash
2010 Communications of the ACM  
68 communications of th e ac m | j u n e 2 0 1 0 | vo l . 5 3 | n o. 6 contributed articles DATA -orienTeD sC i e nT i f iC P ro C es se s depend on fast, accurate analysis of experimental data generated through empirical observation and simulation. However, scientists are increasingly overwhelmed by the volume of data produced by their own experiments. With improving instrument precision and the complexity of the simulated models, data overload promises to only get worse. The inefficiency of
more » ... isting database management systems (DBMSs) for addressing the requirements of scientists has led to many application-specific systems. Unlike their general-purpose counterparts, these systems require more resources, hindering reuse of knowledge. Still, the data-management community aspires to generalpurpose scientific data management. Here, we explore the most important requirements of such systems and the techniques being used to address them. Observation and simulation of phenomena are keys for proving scientific theories and discovering facts of ˲ ˲ Floating-point heavy; and ˲ ˲ Low update rates, with most updates append-only. key insights managing the enormous amount of scientific data being collected is the key to scientific progress. though technology allows for the extreme collection rates of scientific data, processing is still performed with stale techniques developed for small data sets; efficient processing is necessary to be able to exploit the value of huge scientific data collections. Proposed solutions also promise to achieve efficient management for almost any other kind of data. Result of seven-trillion-electronvolt collisions (march 30, 2010) in the atLas particle detector on the Large hadron collider at ceRn, hunting for dark matter, new forces, new dimensions, the higgs boson, and ultimately a grand theory to explain all physical phenomena.
doi:10.1145/1743546.1743568 fatcat:vw57d23aorchtntng6jlrccs6y