IA Scholar Query: A conjugate gradient sampling method for nonsmooth optimization.
https://scholar.archive.org/
Internet Archive Scholar query results feedeninfo@archive.orgFri, 23 Sep 2022 00:00:00 GMTfatcat-scholarhttps://scholar.archive.org/help1440A Decomposition Algorithm for Two-Stage Stochastic Programs with Nonconvex Recourse
https://scholar.archive.org/work/aq7tpsey35dwjjigwb3hfbmeb4
In this paper, we have studied a decomposition method for solving a class of nonconvex two-stage stochastic programs, where both the objective and constraints of the second-stage problem are nonlinearly parameterized by the first-stage variable. Due to the failure of the Clarke regularity of the resulting nonconvex recourse function, classical decomposition approaches such as Benders decomposition and (augmented) Lagrangian-based algorithms cannot be directly generalized to solve such models. By exploring an implicitly convex-concave structure of the recourse function, we introduce a novel decomposition framework based on the so-called partial Moreau envelope. The algorithm successively generates strongly convex quadratic approximations of the recourse function based on the solutions of the second-stage convex subproblems and adds them to the first-stage master problem. Convergence under both fixed scenarios and interior samplings is established. Numerical experiments are conducted to demonstrate the effectiveness of the proposed algorithm.Hanyang Li, Ying Cuiwork_aq7tpsey35dwjjigwb3hfbmeb4Fri, 23 Sep 2022 00:00:00 GMTLayered character models for fast physics-based simulation
https://scholar.archive.org/work/e2u77bn53fd65kfxyzs6sycpmq
This thesis presents two different layered character models that are ready to be used in physics-based simulations, in particular they enable convincing character animations in real-time. We start by introducing a two-layered model consisting of rigid bones and an elastic soft tissue layer that is efficiently constructed from a surface mesh of the character and its underlying skeleton. Building on this model, we introduce Fast Projective Skinning, a novel approach for physics-based character skinning. While maintaining real-time performance it overcomes the well-known artifacts of commonly used geometric skinning approaches. It further enables dynamic effects and resolves local and global self-collisions. In particular, our method neither requires skinning weights, which are often expensive to compute or tedious to hand-tune, nor a complex volumetric tessellation, which fails for many real-world input meshes due to self-intersections. By developing a custom-tailored GPU implementation and a high-quality upsampling method, our ap- proach is the first skinning method capable of detecting and handling arbitrary global collisions in real-time. In the second part of the thesis, we extend the idea of a simplified two-layered volumetric model by developing an anatomically plausible three-layered representation of human virtual characters. Starting with an anatomy model of the male and female body, we show how to generate a layered body template for both sexes. It is composed of three surfaces for bones, muscles and skin enclosing the volumetric skeleton, muscles and fat tissues. Utilizing the simple structure of these templates, we show how to fit them to the surface scan of a person in just a few seconds. Our approach includes a data-driven method for estimating the amount of muscle mass and fat mass from a surface scan, which provides more accurate fits to the variety of human body shapes compared to previous approaches. Additionally, we demonstrate how to efficiently embed fine-scale anatomical details, such as high [...]Martin Komaritzan, Technische Universität Dortmundwork_e2u77bn53fd65kfxyzs6sycpmqThu, 22 Sep 2022 00:00:00 GMTA Unified Bregman Alternating Minimization Algorithm for Generalized DC Programming with Applications to Image Processing
https://scholar.archive.org/work/zomrj3gperb55a7xumsox6xkke
In this paper, we consider a class of nonconvex (not necessarily differentiable) optimization problems called generalized DC (Difference-of-Convex functions) programming, which is minimizing the sum of two separable DC parts and one two-block-variable coupled function. To circumvent the nonconvexity and nonseparability of the problem under consideration, we accordingly introduce a Unified Bregman Alternating Minimization Algorithm (UBAMA) by maximally exploiting the favorable DC structure of the objective. Specifically, we first follow the spirit of alternating minimization to update each block variable in a sequential order, which can efficiently tackle the nonseparablitity caused by the coupled function. Then, we employ the Fenchel-Young inequality to approximate the second DC components (i.e., concave parts) so that each subproblem becomes a convex optimization problem, thereby alleviating the computational burden of the nonconvex DC parts. Moreover, each subproblem absorbs a Bregman proximal regularization term, which is usually beneficial for inducing closed-form solutions of subproblems for many cases via choosing appropriate Bregman functions. It is remarkable that our algorithm not only covers some existing algorithms, but also enjoys implementable schemes with easier subproblems than some state-of-the-art first-order algorithms developed for generic nonconvex and nonsmooth optimization problems. Theoretically, we prove that the sequence generated by our algorithm converges to a critical point under the Kurdyka-Łojasiewicz condition. A series of numerical experiments on image data sets demonstrate the superiority of the proposed algorithmic framework over some existing methods.Hongjin He, Zhiyuan Zhangwork_zomrj3gperb55a7xumsox6xkkeThu, 15 Sep 2022 00:00:00 GMTJoint Demosaicing and Fusion of Multiresolution Compressed Acquisitions: Image Formation and Reconstruction Methods
https://scholar.archive.org/work/uxbsgctvbrcmlakv2n5ahu4xgq
Novel optical imaging devices allow for hybrid acquisition modalities such as compressed acquisitions with locally different spatial and spectral resolutions captured by the same focal plane array. In this work, we propose to model a multiresolution compressed acquisition (MRCA) in a generic framework, which natively includes acquisitions by conventional systems such as those based on spectral/color filter arrays, compressed coded apertures, and multiresolution sensing. We propose a model-based image reconstruction algorithm performing a joint demosaicing and fusion (JoDeFu) of any acquisition modeled in the MRCA framework. The JoDeFu reconstruction algorithm solves an inverse problem with a proximal splitting technique and is able to reconstruct an uncompressed image datacube at the highest available spatial and spectral resolution. An implementation of the code is available at https://github.com/danaroth83/jodefu.Daniele Picone and Mauro Dalla Mura and Laurent Condatwork_uxbsgctvbrcmlakv2n5ahu4xgqSat, 10 Sep 2022 00:00:00 GMTDeep Spatial and Tonal Data Optimisation for Homogeneous Diffusion Inpainting
https://scholar.archive.org/work/nrdfknfr3vbq3lht3jw5v4kwuu
Diffusion-based inpainting can reconstruct missing image areas with high quality from sparse data, provided that their location and their values are well optimised. This is particularly useful for applications such as image compression, where the original image is known. Selecting the known data constitutes a challenging optimisation problem, that has so far been only investigated with model-based approaches. So far, these methods require a choice between either high quality or high speed since qualitatively convincing algorithms rely on many time-consuming inpaintings. We propose the first neural network architecture that allows fast optimisation of pixel positions and pixel values for homogeneous diffusion inpainting. During training, we combine two optimisation networks with a neural network-based surrogate solver for diffusion inpainting. This novel concept allows us to perform backpropagation based on inpainting results that approximate the solution of the inpainting equation. Without the need for a single inpainting during test time, our deep optimisation accelerates data selection by more than four orders of magnitude compared to common model-based approaches. This provides real-time performance with high quality results.Pascal Peter, Karl Schrader, Tobias Alt, Joachim Weickertwork_nrdfknfr3vbq3lht3jw5v4kwuuFri, 09 Sep 2022 00:00:00 GMTProximal nested sampling for high-dimensional Bayesian model selection
https://scholar.archive.org/work/hyho6ssia5hxjdh5oofzwt2cwu
Bayesian model selection provides a powerful framework for objectively comparing models directly from observed data, without reference to ground truth data. However, Bayesian model selection requires the computation of the marginal likelihood (model evidence), which is computationally challenging, prohibiting its use in many high-dimensional Bayesian inverse problems. With Bayesian imaging applications in mind, in this work we present the proximal nested sampling methodology to objectively compare alternative Bayesian imaging models for applications that use images to inform decisions under uncertainty. The methodology is based on nested sampling, a Monte Carlo approach specialised for model comparison, and exploits proximal Markov chain Monte Carlo techniques to scale efficiently to large problems and to tackle models that are log-concave and not necessarily smooth (e.g., involving l_1 or total-variation priors). The proposed approach can be applied computationally to problems of dimension O(10^6) and beyond, making it suitable for high-dimensional inverse imaging problems. It is validated on large Gaussian models, for which the likelihood is available analytically, and subsequently illustrated on a range of imaging problems where it is used to analyse different choices of dictionary and measurement model.Xiaohao Cai, Jason D. McEwen, Marcelo Pereyrawork_hyho6ssia5hxjdh5oofzwt2cwuFri, 09 Sep 2022 00:00:00 GMTStochastic Compositional Optimization with Compositional Constraints
https://scholar.archive.org/work/hsjjo6a6nvaoxbex4zzque3z7i
Stochastic compositional optimization (SCO) has attracted considerable attention because of its broad applicability to important real-world problems. However, existing works on SCO assume that the projection within a solution update is simple, which fails to hold for problem instances where the constraints are in the form of expectations, such as empirical conditional value-at-risk constraints. We study a novel model that incorporates single-level expected value and two-level compositional constraints into the current SCO framework. Our model can be applied widely to data-driven optimization and risk management, including risk-averse optimization and high-moment portfolio selection, and can handle multiple constraints. We further propose a class of primal-dual algorithms that generates sequences converging to the optimal solution at the rate of (1/√(N))under both single-level expected value and two-level compositional constraints, where N is the iteration counter, establishing the benchmarks in expected value constrained SCO.Shuoguang Yang, Zhe Zhang, Ethan X. Fangwork_hsjjo6a6nvaoxbex4zzque3z7iFri, 09 Sep 2022 00:00:00 GMTIs Monte Carlo a bad sampling strategy for learning smooth functions in high dimensions?
https://scholar.archive.org/work/dudcxpagujbahesuqxkzgivhiq
This paper concerns the approximation of smooth, high-dimensional functions from limited samples using polynomials. This task lies at the heart of many applications in computational science and engineering – notably, those arising from parametric modelling and uncertainty quantification. It is common to use Monte Carlo (MC) sampling in such applications, so as not to succumb to the curse of dimensionality. However, it is well known this strategy is theoretically suboptimal. There are many polynomial spaces of dimension n for which the sample complexity scales log-quadratically in n. This well-documented phenomenon has led to a concerted effort to design improved, in fact, near-optimal strategies, whose sample complexities scale log-linearly, or even linearly in n. Paradoxically, in this work we show that MC is actually a perfectly good strategy in high dimensions. We first document this phenomenon via several numerical examples. Next, we present a theoretical analysis that resolves this paradox for holomorphic functions of infinitely-many variables. We show that there is a least-squares scheme based on m MC samples whose error decays algebraically fast in m/log(m), with a rate that is the same as that of the best n-term polynomial approximation. This result is non-constructive, since it assumes knowledge of a suitable polynomial space in which to perform the approximation. We next present a compressed sensing-based scheme that achieves the same rate, except for a larger polylogarithmic factor. This scheme is practical, and numerically it performs as well as or better than well-known adaptive least-squares schemes. Overall, our findings demonstrate that MC sampling is eminently suitable for smooth function approximation when the dimension is sufficiently high. Hence the benefits of improved sampling strategies are generically limited to lower-dimensional settings.Ben Adcock, Simone Brugiapagliawork_dudcxpagujbahesuqxkzgivhiqThu, 08 Sep 2022 00:00:00 GMTPDE-Constrained Equilibrium Problems under Uncertainty: Existence, Optimality Conditions and Regularization
https://scholar.archive.org/work/c3zejnd6rncaln2sdfwglw6qiu
In dieser Arbeit werden PDE-beschränkte Gleichgewichtsprobleme unter Unsicherheiten analysiert. Im Detail diskutieren wir eine Klasse von risikoneutralen verallgemeinerten Nash-Gleichgewichtsproblemen sowie eine Klasse von risikoaversen Nash Gleichgewichtsproblemen. Sowohl für die risikoneutralen PDE-beschränkten Optimierungsprobleme mit punktweisen Zustandsschranken als auch für die risikoneutralen verallgemeinerten Nash Gleichgewichtsprobleme wird die Existenz von Lösungen beziehungsweise Nash Gleichgewichten bewiesen und Optimalitätsbedingungen hergeleitet. Die Betrachtung von Ungleichheitsbedingungen an den stochastischen Zustand führt in beiden Fällen zu Komplikationen bei der Herleitung der Lagrange-Multiplikatoren. Nur durch höhere Regularität des stochastischen Zustandes können wir auf die bestehende Optimalitätstheorie für konvexe Optimierungsprobleme zurückgreifen. Die niedrige Regularität des Lagrange-Multiplikators stellt auch für die numerische Lösbarkeit dieser Probleme ein große Herausforderung dar. Wir legen den Grundstein für eine erfolgreiche numerische Behandlung risikoneutraler Nash Gleichgewichtsproblem mittels Moreau-Yosida Regularisierung, indem wir zeigen, dass dieser Regularisierungsansatz konsistent ist. Die Moreau-Yosida Regularisierung liefert eine Folge von parameterabhängigen Nash Gleichgewichtsproblemen und der Grenzübergang im Glättungsparameter zeigt, dass die stationären Punkte des regularisierten Problems gegen ein verallgemeinertes Nash Gleichgewicht des ursprünglich Problems schwach konvergieren. Die Theorie legt also nahe, dass auf der Moreau-Yosida Regularisierung eine numerische Methode aufgebaut werden kann. Darauf aufbauend werden Algorithmen vorgeschlagen, die aufzeigen, wie risikoneutrale PDE-beschränkte Optimierungsprobleme mit punktweisen Zustandsschranken und risikoneutrale PDE-beschränkte verallgemeinerte Nash Gleichgewichtsprobleme gelöst werden können. Für die Modellierung der Risikopräferenz in der Klasse von risikoaversen Nash Gleichgewichtsprobleme verwenden wi [...]Deborah Berwa Gahururu, Reine Und Angewandte Mathematik, Surowiec, Thomas M. (Prof. Dr.)work_c3zejnd6rncaln2sdfwglw6qiuWed, 07 Sep 2022 00:00:00 GMTDISA: A Dual Inexact Splitting Algorithm for Distributed Convex Composite Optimization
https://scholar.archive.org/work/3tgwua26ine2hmver3nthb2guq
This paper proposes a novel dual inexact splitting algorithm (DISA) for the distributed convex composite optimization problem, where the local loss function consists of an L-smooth term and a possibly nonsmooth term which is composed with a linear operator. We prove that DISA is convergent when the primal and dual stepsizes τ, β satisfy 0<τ<2/L and 0<τβ <1. Compared with existing primal-dual proximal splitting algorithms (PD-PSAs), DISA overcomes the dependence of the convergence stepsize range on the Euclidean norm of the linear operator. It implies that DISA allows for larger stepsizes when the Euclidean norm is large, thus ensuring fast convergence of it. Moreover, we establish the sublinear and linear convergence rate of DISA under general convexity and metric subregularity, respectively. Furthermore, an approximate iterative version of DISA is provided, and the global convergence and sublinear convergence rate of this approximate version are proved. Finally, numerical experiments not only corroborate the theoretical analyses but also indicate that DISA achieves a significant acceleration compared with the existing PD-PSAs.Luyao Guo, Xinli Shi, Shaofu Yang, Jinde Caowork_3tgwua26ine2hmver3nthb2guqMon, 05 Sep 2022 00:00:00 GMTSplitting Method for Support Vector Machine with Lower Semi-continuous Loss
https://scholar.archive.org/work/ulhor3dzpfentlw5rnfzkbf2cu
In this paper, we study the splitting method based on alternating direction method of multipliers for support vector machine in reproducing kernel Hilbert space with lower semi-continuous loss function. If the loss function is lower semi-continuous and subanalytic, we use the Kurdyka-Lojasiewicz inequality to show that the iterative sequence induced by the splitting method globally converges to a stationary point. The numerical experiments also demonstrate the effectiveness of the splitting method.Mingyu Mo, Qi Yework_ulhor3dzpfentlw5rnfzkbf2cuFri, 26 Aug 2022 00:00:00 GMTDecentralized Optimization with Distributed Features and Non-Smooth Objective Functions
https://scholar.archive.org/work/g3bomyucdvgqdigb7pal23c4aq
We develop a new consensus-based distributed algorithm for solving learning problems with feature partitioning and non-smooth convex objective functions. Such learning problems are not separable, i.e., the associated objective functions cannot be directly written as a summation of agent-specific objective functions. To overcome this challenge, we redefine the underlying optimization problem as a dual convex problem whose structure is suitable for distributed optimization using the alternating direction method of multipliers (ADMM). Next, we propose a new method to solve the minimization problem associated with the ADMM update step that does not rely on any conjugate function. Calculating the relevant conjugate functions may be hard or even unfeasible, especially when the objective function is non-smooth. To obviate computing any conjugate function, we solve the optimization problem associated with each ADMM iteration in the dual domain utilizing the block coordinate descent algorithm. Unlike the existing related algorithms, the proposed algorithm is fully distributed and does away with the conjugate of the objective function. We prove theoretically that the proposed algorithm attains the optimal centralized solution. We also confirm its network-wide convergence via simulations.Cristiano Gratton, Naveen K. D. Venkategowda, Reza Arablouei, Stefan Wernerwork_g3bomyucdvgqdigb7pal23c4aqTue, 23 Aug 2022 00:00:00 GMTA Survey of ADMM Variants for Distributed Optimization: Problems, Algorithms and Features
https://scholar.archive.org/work/puetokzpozdapdzeibcbkcxa2e
By coordinating terminal smart devices or microprocessors to engage in cooperative computation to achieve systemlevel targets, distributed optimization is incrementally favored by both engineering and computer science. The well-known alternating direction method of multipliers (ADMM) has turned out to be one of the most popular tools for distributed optimization due to many advantages, such as modular structure, superior convergence, easy implementation and high flexibility. In the past decade, ADMM has experienced widespread developments. The developments manifest in both handling more general problems and enabling more effective implementation. Specifically, the method has been generalized to broad classes of problems (i.e.,multi-block, coupled objective, nonconvex, etc.). Besides, it has been extensively reinforced for more effective implementation, such as improved convergence rate, easier subproblems, higher computation efficiency, flexible communication, compatible with inaccurate information, robust to communication delays, etc. These developments lead to a plentiful of ADMM variants to be celebrated by broad areas ranging from smart grids, smart buildings, wireless communications, machine learning and beyond. However, there lacks a survey to document those developments and discern the results. To achieve such a goal, this paper provides a comprehensive survey on ADMM variants. Particularly, we discern the five major classes of problems that have been mostly concerned and discuss the related ADMM variants in terms of main ideas, main assumptions, convergence behaviors and main features. In addition, we figure out several important future research directions to be addressed. This survey is expected to work as a tutorial for both developing distributed optimization in broad areas and identifying existing theoretical research gaps.Yu Yang, Xiaohong Guan, Qing-Shan Jia, Liang Yu, Bolun Xu, Costas J. Spanoswork_puetokzpozdapdzeibcbkcxa2eTue, 23 Aug 2022 00:00:00 GMTComplexity of Inexact Proximal Point Algorithm for minimizing convex functions with Holderian Growth
https://scholar.archive.org/work/7r7gomcfs5ep5blcuhupkq7cla
Several decades ago the Proximal Point Algorithm (PPA) started to gain a long-lasting attraction for both abstract operator theory and numerical optimization communities. Even in modern applications, researchers still use proximal minimization theory to design scalable algorithms that overcome nonsmoothness. Remarkable works as established tight relations between the convergence behaviour of PPA and the regularity of the objective function. In this manuscript we derive nonasymptotic iteration complexity of exact and inexact PPA to minimize convex functions under γ-Holderian growth: log(1/ϵ) (for γ∈ [1,2]) and 1/ϵ^γ - 2 (for γ > 2). In particular, we recover well-known results on PPA: finite convergence for sharp minima and linear convergence for quadratic growth, even under presence of deterministic noise. Moreover, when a simple Proximal Subgradient Method is recurrently called as an inner routine for computing each IPPA iterate, novel computational complexity bounds are obtained for Restarting Inexact PPA. Our numerical tests show improvements over existing restarting versions of the Subgradient Method.Andrei Patrascu, Paul Iroftiwork_7r7gomcfs5ep5blcuhupkq7claSun, 21 Aug 2022 00:00:00 GMTSurvey of Methods for Solving Systems of Nonlinear Equations, Part II: Optimization Based Approaches
https://scholar.archive.org/work/5tbj5457gnfddca7ei7i6ix6ry
This paper presents a comprehensive survey of methods which can be utilized to search for solutions to systems of nonlinear equations (SNEs). Our objectives with this survey are to synthesize pertinent literature in this field by presenting a thorough description and analysis of the known methods capable of finding one or many solutions to SNEs, and to assist interested readers seeking to identify solution techniques which are well suited for solving the various classes of SNEs which one may encounter in real world applications. To accomplish these objectives, we present a multi-part survey. In part one, we focused on root-finding approaches which can be used to search for solutions to a SNE without transforming it into an optimization problem. In part two, we introduce the various transformations which have been utilized to transform a SNE into an optimization problem, and we discuss optimization algorithms which can then be used to search for solutions. We emphasize the important characteristics of each method, and we discuss promising directions for future research. In part three, we will present a robust quantitative comparative analysis of methods capable of searching for solutions to SNEs.Ilias S. Kotsireas, Panos M. Pardalos, Alexander Semenov, William T. Trevena, Michael N. Vrahatiswork_5tbj5457gnfddca7ei7i6ix6ryWed, 17 Aug 2022 00:00:00 GMTOn the robust isolated calmness of a class of nonsmooth optimizations on Riemannian manifolds and its applications
https://scholar.archive.org/work/f4rg6bugzfewjjtixr4nfqeama
This paper studies the robust isolated calmness property of the KKT solution mapping of a class of nonsmooth optimization problems on Riemannian manifold. The manifold version of the Robinson constraint qualification, the strict Robinson constraint qualification, and the second order conditions are defined and discussed. We show that the robust isolated calmness of the KKT solution mapping is equivalent to the M-SRCQ and M-SOSC conditions. Furthermore, under the above two conditions, we show that the Riemannian augmented Lagrangian method has a local linear convergence rate. Finally, we verify the proposed conditions and demonstrate the convergence rate on two minimization problems over the sphere and the manifold of fixed rank matrices.Yuexin Zhou, Chenglong Bao, Chao Dingwork_f4rg6bugzfewjjtixr4nfqeamaTue, 16 Aug 2022 00:00:00 GMTA Novel Hybrid Method to Predict PM2.5 Concentration Based on the SWT-QPSO-LSTM Hybrid Model
https://scholar.archive.org/work/s3daa5zl4vdevouwfdbry7k6rm
PM2.5 concentration is an important indicator to measure air quality. Its value is affected by meteorological factors and air pollutants, so it has the characteristics of nonlinearity, irregularity, and uncertainty. To accurately predict PM2.5 concentration, this paper proposes a hybrid prediction system based on the Synchrosqueezing Wavelet Transform (SWT) method, Quantum Particle Swarm Optimization (QPSO) algorithm, and Long Short-Term Memory (LSTM) model. First, the original data are denoised by the SWT method and taken as the input of the prediction model. Then, the main parameters of the LSTM model are optimized by global search based on the QPSO algorithm, which solves the problems of slow convergence and local extremum of traditional parameter training algorithms. Finally, the PM2.5 daily concentration data of Chengdu, Shijiazhuang, Shenyang, and Wuhan are predicted by the proposed SWT-QPSO-LSTM model, and the prediction results are compared with those of single prediction models and hybrid prediction models. The experimental results show that the proposed model achieves higher prediction precision and lower prediction error than other models.Meng Du, Yixin Chen, Yang Liu, Hang Yin, Sheng Duwork_s3daa5zl4vdevouwfdbry7k6rmTue, 16 Aug 2022 00:00:00 GMTOn Efficient and Scalable Computation of the Nonparametric Maximum Likelihood Estimator in Mixture Models
https://scholar.archive.org/work/wd76ckshsraknmvo66622zec5m
In this paper we study the computation of the nonparametric maximum likelihood estimator (NPMLE) in multivariate mixture models. Our first approach discretizes this infinite dimensional convex optimization problem by fixing the support points of the NPMLE and optimizing over the mixture proportions. In this context we propose, leveraging the sparsity of the solution, an efficient and scalable semismooth Newton based augmented Lagrangian method (ALM). Our algorithm beats the state-of-the-art methods and can handle n ≈ 10^6 data points with m ≈ 10^4 support points. Our second procedure, which combines the expectation-maximization (EM) algorithm with the ALM approach above, allows for joint optimization of both the support points and the probability weights. For both our algorithms we provide formal results on their (superlinear) convergence properties. The computed NPMLE can be immediately used for denoising the observations in the framework of empirical Bayes. We propose new denoising estimands in this context along with their consistent estimates. Extensive numerical experiments are conducted to illustrate the effectiveness of our methods. In particular, we employ our procedures to analyze two astronomy data sets: (i) Gaia-TGAS Catalog containing n ≈ 1.4 × 10^6 data points in two dimensions, and (ii) the d=19 dimensional data set from the APOGEE survey with n ≈ 2.7 × 10^4.Yangjing Zhang, Ying Cui, Bodhisattva Sen, Kim-Chuan Tohwork_wd76ckshsraknmvo66622zec5mTue, 16 Aug 2022 00:00:00 GMTAccelerated and instance-optimal policy evaluation with linear function approximation
https://scholar.archive.org/work/u6fh37f3k5exxlay2e2gmmy37m
We study the problem of policy evaluation with linear function approximation and present efficient and practical algorithms that come with strong optimality guarantees. We begin by proving lower bounds that establish baselines on both the deterministic error and stochastic error in this problem. In particular, we prove an oracle complexity lower bound on the deterministic error in an instance-dependent norm associated with the stationary distribution of the transition kernel, and use the local asymptotic minimax machinery to prove an instance-dependent lower bound on the stochastic error in the i.i.d. observation model. Existing algorithms fail to match at least one of these lower bounds: To illustrate, we analyze a variance-reduced variant of temporal difference learning, showing in particular that it fails to achieve the oracle complexity lower bound. To remedy this issue, we develop an accelerated, variance-reduced fast temporal difference algorithm (VRFTD) that simultaneously matches both lower bounds and attains a strong notion of instance-optimality. Finally, we extend the VRFTD algorithm to the setting with Markovian observations, and provide instance-dependent convergence results. Our theoretical guarantees of optimality are corroborated by numerical experiments.Tianjiao Li, Guanghui Lan, Ashwin Pananjadywork_u6fh37f3k5exxlay2e2gmmy37mSun, 14 Aug 2022 00:00:00 GMTA Novel Regularization Approach to Fair ML
https://scholar.archive.org/work/mvdmc3xdm5gjphructl6o4lnji
A number of methods have been introduced for the fair ML issue, most of them complex and many of them very specific to the underlying ML moethodology. Here we introduce a new approach that is simple, easily explained, and potentially applicable to a number of standard ML algorithms. Explicitly Deweighted Features (EDF) reduces the impact of each feature among the proxies of sensitive variables, allowing a different amount of deweighting applied to each such feature. The user specifies the deweighting hyperparameters, to achieve a given point in the Utility/Fairness tradeoff spectrum. We also introduce a new, simple criterion for evaluating the degree of protection afforded by any fair ML method.Norman Matloff, Wenxi Zhangwork_mvdmc3xdm5gjphructl6o4lnjiSat, 13 Aug 2022 00:00:00 GMT