Creating reproducible testing and parameterization is a consistent challenge in research because it is hard to do (Koteska, Pejov, & Mishev, 2015) . Researchers often don't have the bandwidth to think about tests, and consequently, creating tools for this use case is often overlooked. GridTest ("GridTest on GitHub," 2020) is a library that specializes in generating parameter grids. The grids are most obviously used for testing, but can extend to other use cases. In the context of testing,

more »

... st makes it easy to discover functions, classes, and arguments for Python scripts or modules, and then generate a template to easily populate. Outside of testing, grids can be created that are version controlled, programatically defined with functions, and easy to interact with from the command line or Python interpreter. Grids can be used with tests that can further be parameterized and configured to collect metrics for each case run. Both grid and test specifications are stored in a simple YAML configuration that the library helps to generate, and features include interactive debugging, interactive report generation, and provided metrics (Python decorators) that can assist with research.