194 Hits in 7.8 sec

A framework for bilevel optimization that enables stochastic and global variance reduction algorithms [article]

Mathieu Dagréou, Pierre Ablin, Samuel Vaiter, Thomas Moreau
2022 arXiv   pre-print
This is the first stochastic algorithm for bilevel optimization that verifies either of these properties. Numerical experiments validate the usefulness of our method.  ...  The simplicity of our approach allows us to develop global variance reduction algorithms, where the dynamics of all variables is subject to variance reduction.  ...  Acknowledgements We thank Othmane Sebbouh, Zaccharie Ramzi and Benoit Malézieux for their precious comments. The authors acknowledge the support of the ANER RAGA BFC.  ... 
arXiv:2201.13409v1 fatcat:kbtnjcsfyjd5po6jqrfag42ixa

A review of recent advances in global optimization

C. A. Floudas, C. E. Gounaris
2008 Journal of Global Optimization  
with grey box/nonfactorable models, and bilevel nonlinear optimization.  ...  This paper presents an overview of the research progress in deterministic global optimization during the last decade (1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006) (2007) (2008).  ...  Acknowledgements Christodoulos A.  ... 
doi:10.1007/s10898-008-9332-8 fatcat:72fpfq72hrdzhf6mxqyc6ssezm

Probabilistic Bilevel Coreset Selection

Xiao Zhou, Renjie Pi, Weizhong Zhang, Yong Lin, Zonghao Chen, Tong Zhang
2022 International Conference on Machine Learning  
In this work, for the first time we propose a continuous probabilistic bilevel formulation of coreset selection by learning a probablistic weight for each training sample.  ...  The overall objective is posed as a bilevel optimization problem, where 1) the inner loop samples coresets and train the model to convergence and 2) the outer loop updates the sample probability progressively  ...  These methods only work for traditional models and can not be directly adopted for DNNs. (Borsos et al., 2020) proposed a bilevel optimization framework for coreset selection for DNN.  ... 
dblp:conf/icml/ZhouPZLCZ22 fatcat:jqqi4o422bdanprmuphwi6yq3y

FEDNEST: Federated Bilevel, Minimax, and Compositional Optimization [article]

Davoud Ataee Tarzanagh, Mingchen Li, Christos Thrampoulidis, Samet Oymak
2022 arXiv   pre-print
We establish provable convergence rates for FEDNEST in the presence of heterogeneous data and introduce variations for bilevel, minimax, and compositional optimization.  ...  However, many contemporary ML problems -- including adversarial robustness, hyperparameter tuning, and actor-critic -- fall under nested bilevel programming that subsumes minimax and compositional optimization  ...  We make multiple algorithmic and theoretical contributions summarized below. • The variance reduction of FEDINN enables robustness in the sense that local models converge to the globally optimal inner  ... 
arXiv:2205.02215v2 fatcat:vs6vcmftrnasbhpm2jyun7tuka

A Single-Timescale Method for Stochastic Bilevel Optimization [article]

Tianyi Chen, Yuejiao Sun, Quan Xiao, Wotao Yin
2022 arXiv   pre-print
This paper develops a new optimization method for a class of stochastic bilevel problems that we term Single-Timescale stochAstic BiLevEl optimization (STABLE) method.  ...  To the best of our knowledge, this is the first bilevel optimization algorithm achieving the same order of sample complexity as the stochastic gradient descent method for the single-level stochastic optimization  ...  Chen and Q. Xiao was partially supported by and the Rensselaer-IBM AI Research Collaboration (, part of the IBM AI Horizons Network ( and NSF 2134168.  ... 
arXiv:2102.04671v4 fatcat:ytan7jyv2bavdotkwayx7s3354

Integrating process design and control using reinforcement learning [article]

Steven Sachio, Max Mowbray, Maria Papathanasiou, Ehecatl Antonio del Rio-Chanona, Panagiotis Petsagkourakis
2021 arXiv   pre-print
For this, one can formulate a bilevel optimization problem, with the design as the outer problem in the form of a mixed-integer nonlinear program (MINLP) and a stochastic optimal control as the inner problem  ...  Therefore, to optimize a process' design, one must address design and control simultaneously.  ...  Furthermore, there is no guarantee that the solution is global, which might be important in some bilevel programming instances [15] , however, guaranteeing global optimality in dynamic optimization is  ... 
arXiv:2108.05242v1 fatcat:cm4r4jiccjgxhnvqq54yu5sihe

Stochastic Bilevel Program for Optimal Coordinated Energy Trading of an EV Aggregator

Yelena Vardanyan, Henrik Madsen
2019 Energies  
This work proposes a stochastic bilevel optimization problem based on the Stackelberg game to create price incentives that generate optimal trading plans for an EV aggregator in day-ahead, intra-day and  ...  The final model is a stochastic convex problem which can be solved efficiently to determine the global optimality. Illustrative results are reported based on a small case with two vehicles.  ...  To enable the access of this type of small-scale flexibility to a large Even though in an EV aggregation context, where a stochastic bilevel optimization approach is taken in which different players act  ... 
doi:10.3390/en12203813 fatcat:5l6qfnchyrastnr557bzg2ek6y

A Primal-Dual Approach to Bilevel Optimization with Multiple Inner Minima [article]

Daouda Sow, Kaiyi Ji, Ziwei Guan, Yingbin Liang
2022 arXiv   pre-print
In this paper, we adopt a reformulation of bilevel optimization to constrained optimization, and solve the problem via a primal-dual bilevel optimization (PDBO) algorithm.  ...  Existing algorithms designed for such a problem were applicable to restricted situations and do not come with a full guarantee of convergence.  ...  Many stochastic bilevel algorithms have been proposed recently via stochastic gradients Ghadimi & Wang (2018) ; Hong et al. (2020) ; , and variance reduction and momentum Chen et al. (2021) ; Khanduri  ... 
arXiv:2203.01123v2 fatcat:gthots22xzfu7lhqu67g7a2dga

ADDS: Adaptive Differentiable Sampling for Robust Multi-Party Learning [article]

Maoguo Gong, Yuan Gao, Yue Wu, A.K.Qin
2021 arXiv   pre-print
In this paper, we propose a novel adaptive differentiable sampling framework (ADDS) for robust and communication-efficient multi-party learning.  ...  Distributed multi-party learning provides an effective approach for training a joint model with scattered data under legal and practical constraints.  ...  data distribution, and it enables the efficient bilevel optimization of model parameters and architecture using gradient descent.  ... 
arXiv:2110.15522v1 fatcat:3zjwlju64ve7jj4uzylnbcbb7u

Threats to Training: A Survey of Poisoning Attacks and Defenses on Machine Learning Systems

Zhibo Wang, Jingjing Ma, Xue Wang, Jiahui Hu, Zhan Qin, Kui Ren
2022 ACM Computing Surveys  
In this survey, we summarize and categorize existing attack methods and corresponding defenses, as well as demonstrate compelling application scenarios, thus providing a unified framework to analyze poisoning  ...  Our ultimate motivation is to provide a comprehensive and self-contained survey of this growing field of research and lay the foundation for a more standardized approach to reproducible studies.  ...  Bilevel optimization provide a uniied paradigm for data poisoning.  ... 
doi:10.1145/3538707 fatcat:pcxpqbsrgzgidb7ngrcb5ggeoa

Data Summarization via Bilevel Optimization [article]

Zalán Borsos, Mojmír Mutný, Marco Tagliasacchi, Andreas Krause
2021 arXiv   pre-print
In this work, we propose a generic coreset construction framework that formulates the coreset selection as a cardinality-constrained bilevel optimization problem.  ...  Coresets are weighted subsets of the data that provide approximation guarantees for the optimization objective.  ...  SNSF grant 407540_167212 through the NRP 75 Big Data program, by the European Research Council (ERC) under the European Union's Horizon 2020 research, innovation programme grant agreement No 815943, and  ... 
arXiv:2109.12534v1 fatcat:f5yewtrb3nehfcs2s5i6jxbfgy

A Survey on Participant Selection for Federated Learning in Mobile Networks [article]

Behnaz Soltani, Venus Haghighi, Adnan Mahmood, Quan Z. Sheng, Lina Yao
2022 arXiv   pre-print
In this paper, we provide a review of participant selection techniques for FL. First, we introduce FL and highlight the main challenges during participant selection.  ...  Federated Learning (FL) is an efficient distributed machine learning paradigm that employs private datasets in a privacy-preserving manner.  ...  They design a stochastic optimization problem that aims to minimize a weighted sum of the convergence bound and communication time.  ... 
arXiv:2207.03681v1 fatcat:yi624z2ggvbv5jl4gs6o4xpvaa

An analytics-based heuristic decomposition of a bilevel multiple-follower cutting stock problem

Adejuyigbe O. Fajemisin, Laura Climent, Steven D. Prestwich
2021 OR spectrum  
We show that current approaches for solving multiple-follower problems are unsuitable for our new class of problems and instead we propose a novel analytics-based heuristic decomposition approach.  ...  AbstractThis paper presents a new class of multiple-follower bilevel problems and a heuristic approach to solving them.  ...  Lu et al. (2006) presents a general framework and solutions for these problems.  ... 
doi:10.1007/s00291-021-00638-9 pmid:34776568 pmcid:PMC8550664 fatcat:5ciwjzhmfraftly2xtyoukiqmi

Functional Bipartite Ranking: a Wavelet-Based Filtering Approach [article]

Stéphan Clémençon, Marine Depecker
2013 arXiv   pre-print
the objective is to learn a scoring function s with optimal ROC curve.  ...  Given a training set of independent realizations of a (possibly sampled) second-order random function with a (locally) smooth autocorrelation structure and to which a binary label is randomly assigned,  ...  Vayatis for helpful and encouraging comments and Prof. A. Gramfort for providing them with the BCI dataset used in the present article.  ... 
arXiv:1312.5066v1 fatcat:rgk5j36og5enpaoebzf2t4sf64

Bilevel Optimization for Machine Learning: Algorithm Design and Convergence Analysis [article]

Kaiyi Ji
2021 arXiv   pre-print
We finally propose new stochastic bilevel optimization algorithms with lower complexity and higher efficiency in practice.  ...  Bilevel optimization has become a powerful framework in various machine learning applications including meta-learning, hyperparameter optimization, and network architecture search.  ...  For algorithm-based bilevel optimization, we first develop a new theoretical framework for analyzing MAML for two types of objective functions that are of interest in practice: (a) resampling case (e.g  ... 
arXiv:2108.00330v1 fatcat:eczdcyzovrcoplozdo6zfdrth4
« Previous Showing results 1 — 15 out of 194 results