Severity-Sensitive Robustness Analysis in Normative Systems [chapter]

Luca Gasparini, Timothy J. Norman, Martin J. Kollingbaum, Liang Chen
2015 Lecture Notes in Computer Science  
In multi-agent systems, norms specify ideal behaviour. Agents, however, are autonomous, and may fail to comply with the ideal. Contrary to Duty obligations can be used to specify reparational behaviour that mitigates the effects of a violation. In addition to specifying reparational behaviours, it is important to understand how robust a system is against possible violations. Depending on what kind of system property we want to preserve, non-compliance with different norms may be of varying
more » ... ity. We propose a method for analysing robustness of normative systems, with support for Contrary to Duty obligations. We introduce violation severity as a concept orthogonal to reparational behaviour and specify it by means of a partial order between norms. We use this severity partial order, together with normative specifications, to rank the possible worlds from the most to the least compliant. In this way, we are able to use model checking to analyse robustness to a certain severity, or whether it is possible to achieve a certain goal, without violating any norm of a given severity. committing a murder (gently or not!) is equivalent to a series of 50 robberies? Penalties for norm violation are imposed post hoc, typically by authorities. Violations may even be excused if the alternative would have been less desirable; e.g. an under-cover policeman choosing between engaging in robberies or commiting a murder to gain trust. Where norms capture the ideal, agents operate autonomously and, hence, their actual behaviour may violate norms. Norm violations may be accidental, due to unanticipated consequences of activities, or deliberate, for example, in order to achieve a goal that would not be possible otherwise. It is, therefore, important to account for and consider the consequences of violations. One way of addressing this issue is to define Contrary To Duty (CTD) obligations. These are structures that describe what an agent should do when a violation occurs. CTD obligations can be used to define a behaviour that mitigates the effects of a violation. In traditional deontic logic frameworks, CTD obligations often lead to inconsistencies [2] . For this reason, a number of logics have been proposed to capture and correctly reason about CTD obligations [7, 11, 12] . In addition to specifying behaviours that may mitigate the effects of a violation through CTD obligations, it is important to understand how robust a normative system is to potential future violations. For example, we may want to determine if certain desired properties are preserved even if a subset of agents in the system fail to comply with the ideal. Ågotnes et al. [1] introduced the idea of verifying robustness of normative systems. They developed a logic, Norm Compliance CTL (NCCTL), for the definition of robustness-related properties. In their model the transitions between possible worlds of a Kripke structure are divided into those allowed (green) and forbidden (red), according to a normative specification. Using NCCTL it is possible to specify properties such as "if a subset of agents comply with the normative system (i.e. do not activate any forbidden transition), it is guaranteed that a certain (un)desired property will (not) hold". In related work, Kazmierczak et al. [9] developed a model checking tool (NorMC) that enables the verification of a NCCTL property for a specified model. Model checking [3] is a formal verification technique that, given a model specification, and some properties, determines whether these properties hold. Properties can be specified using various temporal logic formalisms, such as Linear Temporal Logic (LTL) and Computation Tree Logic (CTL) (or its extension CTL * ). One open question in robustness analysis of normative systems, however, is how to reason about (non) compliance of CTD obligations. Moreover, we believe that, when analysing the robustness of normative systems, it is important to take into account the severity of violations. Our idea is based on the observation that, if our objective is to preserve certain safety properties of a system, some norms are more important than others. In fact, while a system could accept a number of violations of a certain kind, some properties might cease to hold even with only one (more severe) violation of another kind. Our aim is to develop methods to reason about the robustness of normative systems, taking into consideration both violation severity and CTD obligations.
doi:10.1007/978-3-319-25420-3_5 fatcat:qmkbx6kdn5fwbl464ngswzumqu