Security Benchmarks for Web Serving Systems
2014 IEEE 25th International Symposium on Software Reliability Engineering
The assessment of the security level of computer systems in a standardized and regular manner (security benchmarking) has become a very relevant subject, especially for those who use computer systems to support critical business missions or to store confidential information. The concern about computer-based system security is totally justified: systems have become increasingly complex, interconnected, and pervasive, and their security have been threatened by many types of attacks. These attacks
... are unavoidable, as the root causes for them are tied up to human aspects that cannot be removed (intention to cause harm, intention to steal information, etc.), and the losses attacks can cause to their targets (when successful) can be very significant. This scenario of attack inevitability has led companies and governments to invest massively in the development of regulations and mechanisms aimed at the improvement of the security of computer systems (e.g., training developer teams, rapidly solving discovered vulnerabilities, using tools to detect and prevent attacks). Despite these efforts, successful attacks continue to happen, showing that computer systems remain insecure. This is why end-users, system administrators, and systems integrators (to mention just a few classes of users) consider security as an important decision factor when choosing which system to buy and use. These individuals are looking for the means to assess and compare the security of functionally-similar systems/components that will enable them to make a decision taking into account the assessment of security risk. This thesis presents a novel, reproducible, risk-based methodology to benchmark the security of software-based systems. This is a generic methodology that can be instantiated to any class of software-based system. Our benchmark methodology uses the notion of risk in a quantifiable way to measure the security of systems, with a single security metric (SBench) to simplify the comparison of different systems (or different configurations of the same system), enabling users and system integrators to identify and select the most secure one, allowing as well the breakdown of this single metric for more detailed analysis. Our methodology follows the approach of benchmarks proposed in the field of performance and dependability, containing elements such as metrics, workload, and experimental setup, and defining a comprehensive set of procedures and rules to ensure the compliance with key properties such as repeatability. Our security benchmark methodology cover the two complementary views of a 8 given system concerning security: the first takes into account concrete vulnerabilities effectively existing for that system (measures what is already known), and the second estimates the effects of possible yet-to-discover vulnerabilities (and, in fact, many attacks are based on previously unknown vulnerabilities). In fact, these views correspond to the two parts of our benchmark methodology: the static and the dynamic. The static part corresponds to a static analysis of the target system and uses the knowledge about the impact and exploitability of known vulnerabilities discovered for that component or system. The dynamic part corresponds to an experimental analysis of the system in runtime operation when subjected to attacks, while observing the behavior of the system in the presence of these attacks. The combination of the results of these two parts forms the security benchmark measure that enables users, administrators, integrators, and security specialists to identify the most secure among functionally equivalent software systems. This thesis also exemplifies how to apply our security benchmark methodology for a particular and widely used system class (web serving systems), also describing the tools implemented to speed up the execution of the security benchmark. Due to their role in society and exposure to public at general, web serving systems are constantly targeted by attacks, making the implementation of a security benchmark for web serving system a very pertinent contribution. This thesis presents case studies that demonstrate the feasibility, the usefulness and the validity of our security benchmark. Following our methodology, end-users will be able to estimate the security risk of given systems and, if needed, use the results to select the most secure one. The fact that our security benchmark methodology is designed to address any class of softwarebased systems, uses the notion of risk in the benchmark metric, applies an experimental approach to stress the security of systems, and provides procedures and rules that can guide the further development of representative security benchmark standards, make us sure that this is an effective and important contribution to both the industry and the academia.