Dynamic server assignment in an extended machine-repair model
Jan-Pieter L. Dorsman, Sandjai Bhulai, Maria Vlasiou
We consider an extension of the classical machine-repair problem. The machines, apart from receiving service from a single repairman, now also supply service themselves to queues of products. The extended model can be viewed as a two-layered queueing network, in which the queues of products in the first layer are generally correlated, due to the fact that the machines have to share the repairman's capacity in the second layer. We are concerned with the dynamic control problem how the repairman
... hould allocate his capacity to the machines at any point in time so that the long-term average (weighted) sum of the queue lengths of the first-layer queues is minimised. Since the optimal policy for the repairman cannot be found analytically due to the correlations in the queue lengths, we propose a near-optimal policy. We do this by combining intuition and results from queueing theory with techniques from Markov decision theory. Specifically, we study the relative value functions for several policies for which the model can be decomposed in less complicated subsystems, and we combine the results with the classical one-step policy improvement algorithm. The resulting policy is easy to apply, scalable in the number of machines and is shown to be highly accurate over a wide range of parameter settings. Introduction In this paper, we study a queueing model that consists of two layers. The first layer of the model contains two queues of products, see Figure 1 . Each of these queues is served by its own machine. At any point in time, a machine is subject to failure, irrespective of the number of products in each of the queues. When a failure occurs, the service of a product in progress is interrupted. Upon failure, the machine temporarily stops fulfilling its server role in the first layer and becomes a customer in the second layer of the model. The second layer of the model consists of a single repairman capable of repairing failed machines. When the repair of a machine in the second layer has finished, the machine assumes its server role in the first layer again. This model has wide applicability. Evidently, it has immediate applications in manufacturing systems. However, this model is also of interest in less obvious application areas, such as telecommunication systems. For instance, this model occurs naturally in the modelling of middleware technology, where multi-threaded application servers compete for access to shared object code. Access of threads to the object code is typically handled by a portable object adapter (POA) that serialises the threads trying to access a shared object. During the complete duration of the serialisation and execution time a thread typically remains blocked, and upon finalising the execution, the thread is de-activated and ready to process pending requests  . In this setting, the application servers and the Funded in the framework of the STAR-project "Multilayered queueing systems" by the Netherlands Organization for Scientific Research (NWO). The research of M. Vlasiou is also partly supported by an NWO individual grant through project 632.003.002.