Restmule

Beatriz A. Sanchez, Konstantinos Barmpis, Patrick Neubauer, Richard F. Paige, Dimitrios S. Kolovos
2018 Proceedings of the 15th International Conference on Mining Software Repositories - MSR '18  
Mining data from remote repositories, such as GitHub and StackExchange, involves the execution of requests that can easily reach the limitations imposed by the respective APIs to shield their services from overload and abuse. Therefore, data mining clients are left alone to deal with such protective service policies which usually involves an extensive amount of manual implementation efort. In this work we present RestMule, a framework for handling various service policies, such as limited
more » ... of requests within a period of time and multi-page responses, by generating resilient clients that are able to handle request rate limits, network failures, response caching, and paging in a graceful and transparent manner. As a result, RestMule clients generated from OpenAPI speciications (i.e. standardized REST API descriptors), are suitable for intensive data-fetching scenarios. We evaluate our framework by reproducing an existing repository mining use case and comparing the results produced by employing a popular hand-written client and a RestMule client. CCS CONCEPTS • Information systems → RESTful web services; • Software and its engineering → Reusability; Error handling and recovery;
doi:10.1145/3196398.3196405 dblp:conf/msr/SanchezBNPK08 fatcat:bmndesk2tzcsrjejwb3a5yehua