GAMBIT: A Parameterless Model-Based Evolutionary Algorithm for Mixed-Integer Problems

Krzysztof L. Sadowski, Dirk Thierens, Peter A.N. Bosman
2018 Evolutionary Computation  
Learning and exploiting problem structure is one of the key challenges in optimization. This is especially important for black-box optimization (BBO) where prior structural knowledge of a problem is not available. Existing model-based Evolutionary Algorithms (EAs) are very efficient at learning structure in both the discrete, and in the continuous domain. In this paper, discrete and continuous model-building mechanisms are integrated for the Mixed-Integer (MI) domain, comprising discrete and
more » ... tinuous variables. We revisit a recently introduced model-based evolutionary algorithm for the MI domain, the Genetic Algorithm for Model-Based mixed-Integer opTimization (GAMBIT). We extend GAMBIT with a parameterless scheme that allows for practical use of the algorithm without the need to explicitly specify any parameters. We furthermore contrast GAMBIT with other model-based alternatives. The ultimate goal of processing mixed dependences explicitly in GAMBIT is also addressed by introducing a new mechanism for the explicit exploitation of mixed dependences. We find that processing mixed dependences with this novel mechanism allows for more efficient optimization. We further contrast the parameterless GAMBIT with Mixed-Integer Evolution Strategies (MIES) and other state-of-the-art MI optimization algorithms from the General Algebraic Modeling System (GAMS) commercial algorithm suite on problems with and without constraints, and show that GAMBIT is capable of solving problems where variable dependences prevent many algorithms from successfully optimizing them. knowledge can be gained, however, by learning useful structural problem features during optimization. Many real-world applications are based on solving problems which contain both discrete and continuous variables. Such problems are known as mixed-integer (MI) problems and are often regarded as particularly difficult to solve. While many optimization techniques exist about discrete or continuous optimization individually, the area of mixed-integer optimization is relatively less explored. Approaches to mixedinteger optimization do, however, exist. Some of them can be found within the General Algebraic Modeling System (GAMS) framework (Bussieck and Meeraus (2004) ). GAMS is a suite of commercial optimizing software, which includes a collection of MI problem solvers. Another alternatives include an extension of the well-known (µ + λ) evolution strategy (ES) for mixed integer problems, MIES (Li et al. (2013)) and the covariance matrix adaptation evolutionary strategy CMA-ES (Hansen (2011)). A Bayesian network based approach has also been introduced with the Mixed-Integer Bayesian Optimization Algorithm (MIBOA) by Emmerich et al. (2008) . While we consider a selection of already existing approaches, this article explores yet another alternative: model-based Evolutionary Algorithms (EAs). There are various ways in which the structure of a problem can be exploited. Through model building, model-based EAs attempt to dynamically capture the problem structure, often by estimating variable dependences. The work on model-based EAs in both the continuous and the discrete domains is extensive, and has been especially successful in black-box settings. The robustness and efficiency that EAs are known for, along with recently published results on combining continuous and discrete model-building EAs, motivates the idea that model-based EAs can be a potentially powerful approach for the MI domain. The main research objective of this paper is to establish whether the capacity of model-based EAs to capture and exploit problem structure extends to the MI domain. In recent work (Sadowski et al. (2014) ) we have introduced the Genetic Algorithm for Model-Based mixed-Integer opTimization (GAMBIT) and suggested that combining model-based EA approaches from the discrete and continuous domains is a promising foundation for dealing with mixed-integer problem landscapes. However GAM-BIT, like many algorithms, requires specification of some parameter settings before it can begin any optimization process. Population size, for example, is an essential parameter in evolutionary computation. Research results are often presented using the optimal population size parameter settings, which are determined empirically. While this approach is academically insightful, it may often be very impractical, or even infeasible to determine the optimal population size. In order to eliminate the need to specify a population size, we modify and apply a population size-free scheme proposed by Harik and Lobo (1999) for the so-called parameterless GA to GAMBIT. Additionally, we introduce a mechanism which allows for the removal of another important parameter from the user input in GAMBIT: the number of clusters to be used. Coupled with the population size-free scheme, this results in a practical mechanism which removes the need to specify any GAMBIT parameters by the user. Parameterless GAMBIT refers to the use of this mechanism with GAMBIT. Previous work on GAMBIT highlights the importance of properly balancing the use of discrete and continuous models. To further motivate the balanced model approach, we extend our previous work on GAMBIT by comparing it to alternative algorithms based on similar model-based approaches, which do not maintain balance between the continuous and discrete models in the same fashion. Instead, they allow 2 Evolutionary Computation Volume x, Number x 1 The source code of both LTGA and iAMaLGaM is available for download online at
doi:10.1162/evco_a_00206 pmid:28207296 fatcat:hhymbqdj3ze6dfiz5d557rj4yu