Black-box optimization methods play an important role in automated design space exploration, but to-date have not been systematically compared on problems from architectural design optimization. This paper presents a quantitative comparison of the three major types of black-box optimization: metaheuristics, direct search, and model-based methods. We compare the performance of one representative algorithm of each type (including a genetic algorithm) on four performance-based design problems of

more »

... rying complexity and characteristics. Our results show that metaheuristics are greatly outperformed whenever evaluating tens of thousands of design candidates is not an option, and suggest direct search and model-based methods as viable and more efficient alternatives.