TEFIS: A single access point for conducting multifaceted experiments on heterogeneous test facilities

M. Yannuzzi, M.S. Siddiqui, A. Sällström, B. Pickering, R. Serral-Gracià, A. Martínez, W. Chen, S. Taylor, F. Benbadis, J. Leguay, E. Borrelli, I. Ormaetxea (+14 others)
2014 Computer Networks  
A few years ago, an experimental facility composed of networking gear and simulation tools was sufficient for testing the main features of a prototype before the final product could be launched to the Internet market. This paradigm has certainly changed, but the lack of platforms enabling the realistic assessment of the different facets of a product, including cross-cutting trials across different testbeds, poses strong limitations for researchers and developers. In light of this, we present an
more » ... open platform that offers a versatile combination of heterogeneous experimental facilities called "TEstbed for Future Internet Services" (TEFIS). TEFIS provides a single access point for conducting cuttingedge experiments on testbeds that supply different capabilities, including testbeds dedicated to network performance, software performance, grid computing, and living labs. We shall show that TEFIS covers the entire life-cycle of a multifaceted experiment, with the advantage that a single testrun can seamlessly execute across different experimental facilities. In order to demonstrate the potential and versatility of the TEFIS platform, we describe the deployment of four distinct experiments and provide a set of results highlighting the benefits of using TEFIS. The experiments described in this article cover: i) the experimentation with an open API called OPENER (which is an open and programmable environment for managing experimentation with SDN applications); ii) an application for skiers and tourists at the Megève ski resort in France; iii) an application that can dynamically adapt the Quality of Experience (QoE) of multimedia services for mobile users; and iv) an augmented reality workspace for remote education and learning purposes based on videoconferencing. would like to assess not only the performance of a distributed application along with the experience obtained by the users, but also the quality and reliability of the software packages that support the application itself. Ideally, experimenters would like to run such tests in "a single place", irrespective of how many different test facilities are required, and they would like to be able to manage their experiments throughout its entire life-cycle. Moreover, experimenters would also like to be able to build upon other experimenters' work, as well as learn from their experiences. This would obviously facilitate the overall experimentation process, and would allow experimenters to contribute with their own work to the existing knowledge base. Our aim in this paper is to present a new platform with the potential to meet these ideals. The "TEstbed for Future Internet Services" (TEFIS) [1] is such a platform and represents a large European effort enabling multifaceted experiments on different test facilities. More specifically, TEFIS offers a versatile combination of heterogeneous experimental facilities through "a single access point", and it is built as an open platform that currently provides seamless access for testing services on six different testbeds, namely: (1) PACAGrid [2] (a grid computing facility for experimenting with computationally intensive applications); (2) ETICS [3] (an e-infrastructure that provides support for testing the entire software development life-cycle); (3) SQS-IMS [4] (an infrastructure for testing protocols and mobile applications over the IP Multimedia Subsystem (IMS) [5]); (4) The Botnia Living Lab [6] (a Living Lab that offers end-user engagement in the design and testing of Future Internet technologies and services); (5) Kyatera [7] (a high-speed optical network for supporting experimentation of technologies with large bandwidth demands); and (6) PlanetLab [8] (a testbed for distributed systems and network research on a large scale over the Internet). By seamlessly combining these different testbeds, TEFIS can cover most of the development and testing activities, such as building and packaging software, system integration, Service-Level Agreement (SLA) dimensioning, largescale deployment and testing, compliance testing, as well as "user-level evaluation" of run-time services. As we shall show, the TEFIS platform provides all the necessary services enabling the management of underlying resources for executing complex experiments on multiple test facilities. In particular, TEFIS handles aspects such as resource management (e.g., resource access, matching and identification of resources that can be activated, etc.), software deployment, measurement-related services for a variety of testbeds, as well as data persistence. One of the key advantages of TEFIS is that this handling is not exposed to experimenters, since the platform abstracts the internals of the different testbeds by means of a flexible interface that enables not only the robust configuration of experiments, but also the automation and repeatability of the same. Overall, this paper makes the following contributions. We describe an open architecture that is sufficiently general as to enable multifaceted experimentation on multiple (and quite different) testbeds. We show that the combination of testbeds is made possible by means of "connectors", which provide the desired level of abstraction for experimenters while facilitating the process of adding and/or removing testbeds as needed. As a proof-of-concept, we describe four distinct experiments that were recently run on TEFIS, and show how they have benefit from the resources offered by the different test facilities. We also examine some of the future challenges faced by experimental platforms such as TEFIS, including a discussion on business and sustainability models allowing the continuity of these platforms. We contend that the TEFIS model can be positioned as an enabler for building a community of experimenters, which can increase their expertise and share their knowledge in order to accelerate, and improve, the design and Borrelli), iormaetxea@sqs.es (I. Ormaetxea), konrad.campowsky@fokus.fraunhofer.de (K. Campowsky),
doi:10.1016/j.bjp.2013.12.030 fatcat:lkoejy5duzgi5lqxch7zeagr2q