587 Hits in 4.2 sec

Delayed Nondeterminism in Continuous-Time Markov Decision Processes [chapter]

Martin R. Neuhäußer, Mariëlle Stoelinga, Joost-Pieter Katoen
2009 Lecture Notes in Computer Science  
We consider continuous-time Markov decision processes (CTMDPs) and define a hierarchy of positional (P) and historydependent (H) schedulers which induce strictly tighter bounds on quantitative properties  ...  In particular, we show that TTP and TAP schedulers allow for delaying nondeterminism for all measures, whereas this does neither hold for TP nor for any TAH scheduler.  ...  In addition, in contrast to TP and TAHOP schedulers, TH, TTH, and TAH schedulers cannot lose information to optimize their decisions. TAH and TAHOP schedulers can also gain information.  ... 
doi:10.1007/978-3-642-00596-1_26 fatcat:y7k7pn23kffalf4fx5tru7eqwa


Mario Bravetti, Holger Hermanns, Joost-Pieter Katoen
2006 Electronical Notes in Theoretical Computer Science  
Continuous-time Markov chains (CTMCs) are a widely used performance evaluation model.  ...  Markov chains are widely used to determine system performance and reliability characteristics. The vast majority of applications considers continuous-time Markov chains (CTMCs).  ...  In fact, it is the class of continuous-time Markov decision chains [10] , a strict superset of CTMCs. Nondeterminism is one of the vital ingredients of process algebra and hence of IMC.  ... 
doi:10.1016/j.entcs.2005.12.108 fatcat:rkd3kxi3i5bvrlw2obndku4264

The Stochastic Quality Calculus [chapter]

Kebin Zeng, Flemming Nielson, Hanne Riis Nielson
2014 Lecture Notes in Computer Science  
The communication delays are expressed by general distributions and the resulting semantics is given in terms of Generalised Semi-Markov Decision Processes.  ...  By restricting the distributions to be continuous and by allowing truly concurrent communication we eliminate the nondeterminism and arrive at Generalised Semi-Markov Processes (GSMPs); further restriction  ...  The semantics of SQC amounts to generalised semi-Markov decision processes.  ... 
doi:10.1007/978-3-662-43376-8_12 fatcat:z4waew7kb5dgznculyh4ccfspu

A Set of Performance and Dependability Analysis Components for CADP [chapter]

Holger Hermanns, Christophe Joubert
2003 Lecture Notes in Computer Science  
Similar to Markov decision processes [20] , the Imc model allows nondeterminism in states, i.e., two identical action transitions leaving the same state.  ...  To achieve this we use the theory of interactive Markov chains [13] , a conservative extension of both process algebra and continuous-time Markov chains (Ctmc for short), the latter being a well-investigated  ... 
doi:10.1007/3-540-36577-x_30 fatcat:psz3672jqjfzvns52sihddnyq4

Compositional Theories of Qualitative and Quantitative Behaviour [chapter]

Ed Brinksma
2003 Lecture Notes in Computer Science  
Process algebras are typically equipped with a formally defined structured operational semantics (SOS [26]) that maps process algebra terms onto labelled transition systems in a compositional manner.  ...  Traditional performance models like Markov chains and queueing networks are widely accepted as simple but effective models in different areas, yet they lack the notion of hierarchical system (de)composition  ...  For more than a decade the author has had the privilege to discuss and work on stochastic process algebraic topics with a number of great colleagues from whom he has learned a lot.  ... 
doi:10.1007/3-540-44919-1_5 fatcat:2xcbvmkuifg33nc4jwfcx5cy4a

Ten Years of Performance Evaluation for Concurrent Systems Using CADP [chapter]

Nicolas Coste, Hubert Garavel, Holger Hermanns, Frédéric Lang, Radu Mateescu, Wendelin Serwe
2010 Lecture Notes in Computer Science  
A prominent example of such specification formalisms is the class of process algebras, which provide abstraction mechanisms to treat system components as black boxes,  ...  Traditional performance models like Markov chains and queueing networks are not easy to apply for large-sized systems, mainly because they lack hierarchical composition and abstraction means.  ...  , Discrete Time Markov Decision Processes, Generative Probabilistic Lts, Reactive Probabilistic Lts, Stratified probabilistic Lts. 5 Continuous Time Markov Chains, Continuous Time Markov Reward Models,  ... 
doi:10.1007/978-3-642-16561-0_18 fatcat:eenulhij7rfe7djagjri4cpaju

GSPNs Revisited: Simple Semantics and New Analysis Algorithms

Joost-Pieter Katoen
2012 2012 12th International Conference on Application of Concurrency to System Design  
This paper considers interactive Markov chains (IMCs), a natural generalization of transition systems and continuous-time Markov chains (CTMCs).  ...  In addition, we will present novel analysis algorithms for expected time and long-run average time objectives of IMCs, i.e., GSPNs.  ...  It recently has been shown that the computation of minimal (or maximal) expected time objectives in IMCs can be reduced to to a non-negative stochastic shortest path (SSP) problem in Markov decision processes  ... 
doi:10.1109/acsd.2012.30 dblp:conf/acsd/Katoen12 fatcat:en7rofnavbfbzkswgvcymewi2a

Trace Semantics for Stochastic Systems with Nondeterminism

Verena Wolf, Christel Baier, Mila Majster-Cederbaum
2006 Electronical Notes in Theoretical Computer Science  
This paper discusses trace semantics of interactive Markov chains (IMCs) that are a generalisation of continuous-time Markov chains.  ...  In IMCs besides probabilistic alternatives there may be a nondeterministic choice between several action-labelled transitions in a state.  ...  Formally, interactive Markov chains are state-transition graphs that combine the features of LTSs with continuous-time Markov chains.  ... 
doi:10.1016/j.entcs.2006.07.019 fatcat:nom4ijfrbzcmzdtbbw3vvuwkfa

Advances and challenges of probabilistic model checking

Marta Kwiatkowska, Gethin Norman, David Parker
2010 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton)  
In this paper, we give a short overview of probabilistic model checking and of PRISM (, currently the leading software tool in this area.  ...  Such systems are found in many domains: probabilistic behaviour may arise, for example, due to failures of unreliable components, communication across lossy media, or through the use of randomisation in  ...  The authors are supported in part by EPSRC projects EP/D07956X and EP/F001096, EU FP7 project CONNECT and ERC Advanced Grant VERIWARE.  ... 
doi:10.1109/allerton.2010.5707120 fatcat:bauounyehvelpe4dwjrobeugfe

Using probabilistic model checking for dynamic power management

Gethin Norman, David Parker, Marta Kwiatkowska, Sandeep Shukla, Rajesh Gupta
2005 Formal Aspects of Computing  
We present an approach to deriving stochastic Dynamic Power Management (DPM) strategies that enables us to design both discrete time and continuous time Markov chain based strategies, in a formal and uniform  ...  Moreover, one can formally prove various probabilistically quantified properties pertaining to buffer sizes, delays, energy usage etc., for each derived strategy.  ...  It supports analysis of three types of probabilistic models: discrete-time Markov chains, continuous-time Markov chains and Markov decision processes.  ... 
doi:10.1007/s00165-005-0062-0 fatcat:vvm3yxvl6rcpndcikddjsxmwl4

Probabilistic Model Checking of the CSMA/CD Protocol Using PRISM and APMC

Marie Duflot, Laurent Fribourg, Thomas Herault, Richard Lassaigne, Frédéric Magniette, Stéphane Messika, Sylvain Peyronnet, Claudine Picaronny
2005 Electronical Notes in Theoretical Computer Science  
We then use the model-checking tool PRISM [30] in order to analyse the resulting Markov decision process for the CSMA/CD protocol.  ...  The underlying Markov decision process then becomes a "fully probabilistic system" (or, in other terms, a Markov chain), and can then be analysed via the tool APMC [13] .  ...  The PTA modelling the system can then be seen as a finite-state Markov decision process [10] .  ... 
doi:10.1016/j.entcs.2005.04.012 fatcat:wxr6oba4jzewbcaa2yzqz7c5wy

MoDeST — A Modelling and Description Language for Stochastic Timed Systems [chapter]

Pedro R. D'Argenio, Holger Hermanns, Joost-Pieter Katoen, Ric Klaren
2001 Lecture Notes in Computer Science  
In addition, MoDeST incorporates means to describe important phenomena such as non-determinism, probabilistic branching, and hard real-time as well as soft real-time (i.e., stochastic) aspects.  ...  The language is influenced by popular and user-friendly specification languages such as Promela, and deals with compositionality in a light-weight process-algebra style.  ...  MC: Continuous time Markov chains are a standard model in contemporary performance evaluation.  ... 
doi:10.1007/3-540-44804-7_6 fatcat:ojdvn4pcm5dy5pi3n4eam3ea4m

Schedulers are no Prophets [chapter]

Arnd Hartmanns, Holger Hermanns, Jan Krčál
2015 Lecture Notes in Computer Science  
Several formalisms for concurrent computation have been proposed in recent years that incorporate means to express stochastic continuous-time dynamics and non-determinism.  ...  In this setting, some obscure phenomena are known to exist, related to the fact that schedulers may yield too pessimistic verification results, since current nondeterminism can surprisingly be resolved  ...  A Markov decision process turns into a Markov chain by fixing a resolution of nondeterminism.  ... 
doi:10.1007/978-3-319-27810-0_11 fatcat:rjczvpwxqbehhpssdgdakpohnu

Game-based Abstraction for Markov Decision Processes

Marta Z. Kwiatkowska, Gethin Norman, David Parker
2006 Third International Conference on the Quantitative Evaluation of Systems - (QEST'06)  
In this paper we present a novel abstraction technique for Markov decision processes (MDPs), which are widely used for modelling systems that exhibit both probabilistic and nondeterministic behaviour.  ...  The key idea behind this approach is to maintain a separation between nondeterminism present in the original MDP and nondeterminism introduced through abstraction, each type being represented by a different  ...  Acknowledgements The authors are supported in part by EPSRC grants GR/S11107 and GR/S46727 and Microsoft Research Cambridge contract MRL 2005-44.  ... 
doi:10.1109/qest.2006.19 dblp:conf/qest/KwiatkowskaNP06 fatcat:azfx2rf7kvgfpaud3n5zonkltu

Page 6564 of Mathematical Reviews Vol. , Issue 2000i [page]

2000 Mathematical Reviews  
In Chapter 4 we introduce the central formalism of this thesis, interactive Markov chains, that arise as an integration of inter- active processes and continuous time Markov chains.  ...  In this dissertation we address the issue of com- positional specification and analysis of continuous time Markov chains.  ... 
« Previous Showing results 1 — 15 out of 587 results