170,149 Hits in 5.6 sec

Software architecture for large-scale, distributed, data-intensive systems

C.A. Mattmann, D.J. Crichton, S.J. Hughes, S.C. Kelly, M. Paul
Proceedings. Fourth Working IEEE/IFIP Conference on Software Architecture (WICSA 2004)  
These data-intensive systems exhibit characteristics which appear fruitful for research from a software engineering, and software architectural focus.  ...  The sheer amount of data produced by modern science research has created a need for the construction and understanding of "data-intensive systems", largescale, distributed systems which integrate information  ...  [14, 15] present a novel architecture for describing large-scale data-intensive information systems, specifically applied to NASA's EOSDIS science domain.  ... 
doi:10.1109/wicsa.2004.1310708 dblp:conf/wicsa/MattmannCHKR04 fatcat:tycazj5yxbfinir5vngtvnddxy

Distribution, Data, Deployment: Software Architecture Convergence in Big Data Systems

Ian Gorton, John Klein
2015 IEEE Software  
Big data systems present many challenges to software architects. In particular, distributed software architectures become tightly coupled to data and deployment architectures.  ...  new specialization for software technology: data-intensive, or big data, software systems. 1 Internet-born organizations such as Google and Amazon are on this revolution's cutting edge, collecting, managing  ...  This material has been approved for public release and unlimited distribution. DM-0000810.  ... 
doi:10.1109/ms.2014.51 fatcat:jz5e44qgo5fxpgcxcunt5vxeve

Software Design and Implementation for MapReduce across Distributed Data Centers

Lizhe Wang, Jie Tao, Yan Ma, Samee U. Khan, Joanna Kołodziej, Dan Chen
2013 Applied Mathematics & Information Sciences  
Recently, the computational requirements for large-scale data-intensive analysis of scientific data have grown significantly.  ...  The MapReduce paradigm has emerged as a highly successful programming model for large-scale data-intensive computing applications.  ...  Figure 1 Architecture Overview of G-Hadoop G-Hadoop Gfarm as a global distributed file system The MapReduce framework for data-intensive applications heavily relies on the underlying distributed file  ... 
doi:10.12785/amis/071l13 fatcat:7ip6kgxvc5dgzcyicdhxpyaxem

Exascale Machines Require New Programming Paradigms and Runtimes

2015 Supercomputing Frontiers and Innovations  
In this article, we explore the shortcomings of existing programming models and runtimes for large-scale computing systems.  ...  This article is structured as follows: the next section describes the requirements from the programmability point of view for extra large-scale systems such as ultrascale systems.  ...  Modeling and simulation of failures in large-scale systems Exascale computing provides a large-scale, heterogeneous distributed computing environment for the processing of demanding jobs.  ... 
doi:10.14529/jsfi150201 fatcat:ozj4czefxrd37j7djcxuukyuee

From Principles to Details: Integrated Framework for Architecture Modelling of Large Scale Software Systems

Andrzej Zalewski, Szymon Kijas
2013 e-Informatica Software Engineering Journal  
We intend to prove that a less abstract modelling framework is needed for the effective modelling of large scale software intensive systems.  ...  recommending practice for the architectural description.  ...  Figure 2 . 2 The overall structure of the integrated architecture model for large scale software systems -Data storage distribution: defines how permanent data is distributed among databases; Our example  ... 
doi:10.5277/e-inf130105 dblp:journals/eInformatica/ZalewskiK13 fatcat:5f44zmtjabf67douc7lz3icz6i

A limits study of benefits from nanostore-based future data-centric system architectures

Jichuan Chang, Parthasarathy Ranganathan, Trevor Mudge, David Roberts, Mehul A. Shah, Kevin T. Lim
2012 Proceedings of the 9th conference on Computing Frontiers - CF '12  
To address the challenges in evaluating such system architectures for distributed systems, we develop and validate a new methodology for large-scale datacentric workloads.  ...  We also discuss and quantify the impact of network bandwidth, software scalability, and power density, and design tradeoffs for future NVM-based data-centric architectures.  ...  Optimizations at the technology, circuit, and systems levels [15] [20] [21] The second challenge is around horizontal scaling required for large-scale distributed systems.  ... 
doi:10.1145/2212908.2212915 dblp:conf/cf/ChangRMRSL12 fatcat:nf4htkrtgze4fd2yevuovipzpq

A Study of Resilient Architecture for Critical Software-Intensive System-of-Systems (Sisos)

Nadeem Akhtar, Malik Muhammad, Nadeem Salamat, Amnah Firdous, Mujtaba Husnain
2016 International Journal of Advanced Computer Science and Applications  
An approach has been proposed for the analysis, design, formal specification and verification of critical Software-intensive System-of-Systems.  ...  The role of critical system-of-systems have become considerably software-intensive. A critical system-of-system has to satisfy correctness properties of liveness and safety.  ...  Objectives The major objective of resilient architecture for software intensive system-of-systems is to develop efficient and robust approach for building resilient large-scale critical system-ofsystems  ... 
doi:10.14569/ijacsa.2016.070834 fatcat:g4kdhh7r7vg63k2dfsebkzjtx4

Energy-Efficient Big Data Analytics in Datacenters [chapter]

Farhad Mehdipour, Hamid Noori, Bahman Javadi
2016 Advances in Computers  
First the datacenter architecture including computing and networking technologies as well as data centers for cloud-based services will be illustrated.  ...  Also, as the scale of the datacenter is increasingly expanding, minimizing energy consumption and operational cost is a vital concern.  ...  FAWN architecture [2] is another solution for building cluster systems for energy-efficient serving massive-scale I/O and data-intensive workloads.  ... 
doi:10.1016/bs.adcom.2015.10.002 fatcat:xpecmdmje5avvphkdrdnyv4fqi

Cloud-based Data-intensive Framework towards fault diagnosis in large-scale petrochemical plants

Zhiqiang Huo, Mithun Mukherjee, Lei Shu, Yuanfang Chen, Zhangbing Zhou
2016 2016 International Wireless Communications and Mobile Computing Conference (IWCMC)  
In this paper, we propose a Cloud-based Data-intensive Framework (CDF) for on-line equipment fault diagnosis system that facilitates the integration and processing of mass sensor data generated from Industrial  ...  , involves heterogeneity and Big Data problems due to large amounts of sensor data with high volume and velocity.  ...  This paper presents a Cloud-based Data-intensive Framework, named as CDF, that supports large-scale equipment fault diagnosis systems in a cloud environment.  ... 
doi:10.1109/iwcmc.2016.7577209 dblp:conf/iwcmc/HuoMSCZ16 fatcat:3ujxzy5lczeg7cg466heawed5q

Data-Intensive Computing Infrastructure Systems for Unmodified Biological Data Analysis Pipelines [chapter]

Lars Ailo Bongo, Edvard Pedersen, Martin Ernstsen
2015 Lecture Notes in Computer Science  
We give an overview of data-intensive computing infrastructure systems, and describe how we have leveraged these for: (i) scalable fault-tolerant computing for large-scale biological data; (ii) incremental  ...  These pipelines must scale to very large datasets, and consequently require parallel and distributed computing.  ...  Thanks to our colleagues at the Tromsø ELIXIR node for their expertise in developing and deploying biological data processing systems.  ... 
doi:10.1007/978-3-319-24462-4_22 fatcat:onjzez7qireirmdc5aj2ajgimm

Technology and Data-Intensive Science in the Beginning of the 21st Century

Philip A. Bernstein, Dave Wecker, Ashok Krishnamurthy, Dinesh Manocha, Jeffrey Gardner, Natali Kolker, Chance Reschke, Jesse Stombaugh, Pamela Vagata, Elizabeth Stewart, Dean Welch, Eugene Kolker
2011 Omics  
This article is a summary of the technology issues and challenges of data-intensive science and cloud computing as discussed in the Data-Intensive Science (DIS) workshop in Seattle,  ...  Office of Cyberinfrastructure, Task Force on Data and Visualization (in press).  ...  Barriers Programming models for distributed systems are hard to use. Exploiting chip-level parallelism is challenging, especially for data-intensive computation.  ... 
doi:10.1089/omi.2011.0013 pmid:21476841 fatcat:kskd6zgztbgijkiuzmjoptvyzy

GLIDE: A Grid-Based Light-Weight Infrastructure for Data-Intensive Environments [chapter]

Chris A. Mattmann, Sam Malek, Nels Beckman, Marija Mikic-Rakic, Nenad Medvidovic, Daniel J. Crichton
2005 Lecture Notes in Computer Science  
To address these limitations, we present GLIDE, a prototype light-weight, data-intensive middleware infrastructure that enables access to the robust data and computational power of the grid on DREAM platforms  ...  The promise of the grid is that it will enable public access and sharing of immense amounts of computational and data resources among a large number of individuals and institutions.  ...  Given the central role software architectures have played in engineering large-scale distributed systems [21] , we hypothesize that their importance will only grow in the even more complex (grid-enabled  ... 
doi:10.1007/11508380_9 fatcat:oqvfkvwhpjeinbokhqhu3nvir4

Affordable Supercomputing for Data Mining Applications

András A. Benczúr
2011 Procedia Computer Science  
A cluster of commodity machines is ideal for data intensive problems.  ...  Business intelligence, e-science and Web mining are rapidly growing sources of extreme large scale problems.  ...  As another initiative, in the LAWA project we will build an Internet-based experimental testbed for large-scale data analytics.  ... 
doi:10.1016/j.procs.2011.09.009 fatcat:jlcdgcgpfjgbvf4uubt27ypcim

MapReduce across Distributed Clusters for Data-intensive Applications

Lizhe Wang, Jie Tao, Holger Marten, Achim Streit, Samee U. Khan, Joanna Kolodziej, Dan Chen
2012 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum  
The MapReduce paradigm has emerged as a highly successful programming model for large-scale data-intensive computing applications.  ...  On the other hand, workflow systems are used for distributed data processing across data centers.  ...  The goal of this research is to advance the MapReduce framework for large-scale distributed computing across multiple data centers with multiple clusters.  ... 
doi:10.1109/ipdpsw.2012.249 dblp:conf/ipps/WangTMSKKC12 fatcat:xnrdnqzpubgm5lm7jwhwejuo5m

Making a case for distributed file systems at Exascale

Ioan Raicu, Ian T. Foster, Pete Beckman
2011 Proceedings of the third international workshop on Large-scale system and application performance - LSAP '11  
At exascale, basic functionality at high concurrency levels will suffer poor performance, and combined with system mean-time-to-failure in hours, will lead to a performance collapse for large-scale heroic  ...  The current architecture of high-end computing systems is decades-old and has persisted as we scaled from gigascales to petascales.  ...  We want to thank our collaborators for the valuable help, feedback, and insight leading up to this work, namely Mike Wilde, Matei Ripeanu, Arthur Barney Maccabe, Marc Snir, Rob Ross, Kamil Iskra, and Alok  ... 
doi:10.1145/1996029.1996034 fatcat:bon3bizokzckhl5ajrzsqt7tqq
« Previous Showing results 1 — 15 out of 170,149 results