Filters








11,713 Hits in 3.2 sec

K-NN active learning under local smoothness assumption [article]

Boris Ndjia Njike, Xavier Siebert
2020 arXiv   pre-print
We design an active learning algorithm with a rate of convergence better than in passive learning, using a particular smoothness assumption customized for k-nearest neighbors.  ...  Unlike previous active learning algorithms, we use a smoothness assumption that provides a dependence on the marginal distribution of the instance space.  ...  Each element of S can be seen as a triplet (X ′ , Y ′ , c) where X ′ is an informative point, Y ′ its inferred label, and c > 0 can be thought as Algorithm 1: k-NN Active Learning under Local Smoothness  ... 
arXiv:2001.06485v2 fatcat:kaunu657ajadxkoq4l7a4dddda

K-nn active learning under local smoothness condition [article]

Boris Ndjia Njike, Xavier Siebert
2021 arXiv   pre-print
We provide a novel active learning algorithm with a rate of convergence better than in passive learning, using a particular smoothness assumption customized for k-nearest neighbors.  ...  Here we outline some of the results that have been obtained, more specifically in a nonparametric setting under assumptions about the smoothness and the margin noise.  ...  Algorithm 1 : 1 k-nn Active Learning under Local Smoothness (KALLS) Input: K = {X 1 , . . . , X w }, n, α, L, δ, C, β, ǫ; Output: 1-nn classifier f n s = 1 ⊲ index of point currently examined S = ∅ ⊲ current  ... 
arXiv:1902.03055v3 fatcat:odlkvmuqf5eo3kc5v4ee2okgmq

Generalized Gradient Learning on Time Series under Elastic Transformations [article]

Brijnesh Jain
2015 arXiv   pre-print
Necessary conditions are presented under which generalized gradient learning on time series is consistent.  ...  For these representations many standard learning algorithms are unavailable. We generalize gradient-based learning algorithms to time series under dynamic time warping.  ...  Thus, the complexity of NN+AHC is O(n 2 N 2 ) in the best and O(n 2 N 2 + N 3 ) in the general case. The NN+KME learns a protoype for each class using k-means under elastic transformations.  ... 
arXiv:1502.04843v2 fatcat:xmgsf3eoezb5tnu653ehtbgkwy

Generalized gradient learning on time series

Brijnesh J. Jain
2015 Machine Learning  
Necessary conditions are sketched under which generalized gradient learning on time series is consistent.  ...  For these representations many standard learning algorithms are unavailable. We generalize gradient-based learning algorithms to time series under dynamic time warping.  ...  Thus, the complexity of NN+AHC is O(n 2 N 2 ) in the best and O(n 2 N 2 + N 3 ) in the general case. The NN+KME learns a protoype for each class using k-means under elastic transformations.  ... 
doi:10.1007/s10994-015-5513-0 fatcat:td2pqa7f7vdpvesytipqhmc2k4

Variance reduction with neighborhood smoothing for local intrinsic dimension estimation

Kevin M. Carter, Alfred O. Hero
2008 Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing  
Local intrinsic dimension estimation has been shown to be useful for many tasks such as image segmentation, anomaly detection, and de-biasing global dimension estimates.  ...  Of particular concern with local dimension estimation algorithms is the high variance for high dimensions, leading to points which lie on the same manifold estimating at different dimensions.  ...  Local Dimension Estimation The k-NN algorithm in itself is a global dimension estimator, i.e. it globally fits the k-NN graph length functional L n and solves (3) over the entire sample space.  ... 
doi:10.1109/icassp.2008.4518510 dblp:conf/icassp/CarterH08 fatcat:ypferreidrbo5lhkbmv6kyb5ru

Locally Adaptive Label Smoothing Improves Predictive Churn

Dara Bahri, Heinrich Jiang
2021 International Conference on Machine Learning  
In this paper, we present several baselines for reducing churn and show that training on soft labels obtained by adaptively smoothing each example's label based on the example's neighboring labels often  ...  We show under mild nonparametric assumptions that for a wide range of k, the k-NN labels uniformly approximates the optimal soft label and when k is tuned optimally, achieves the minimax optimal rate.  ...  Algorithm 1 shows how k-NN label smoothing is applied to deep learning models. Like Bahri et al. (2020) , we perform k-NN on the network's logits layer.  ... 
dblp:conf/icml/BahriJ21 fatcat:dmyvtqanxzd45geotgd7xr4i4y

Locally Adaptive Label Smoothing for Predictive Churn [article]

Dara Bahri, Heinrich Jiang
2021 arXiv   pre-print
In this paper, we present several baselines for reducing churn and show that training on soft labels obtained by adaptively smoothing each example's label based on the example's neighboring labels often  ...  We show under mild nonparametric assumptions that for a wide range of k, the k-NN labels uniformly approximates the optimal soft label and when k is tuned optimally, achieves the minimax optimal rate.  ...  Algorithm 1 shows how k-NN label smoothing is applied to deep learning models. Like Bahri et al. (2020) , we perform k-NN on the network's logits layer.  ... 
arXiv:2102.05140v2 fatcat:d6ak5z2q7zgaxeqti2bt435g6a

Semi-supervised Regression and System Identification, [chapter]

Henrik Ohlsson, Lennart Ljung
2010 Three Decades of Progress in Control Sciences  
We outline a general approach to semi-supervised regression, describe its links to Local Linear Embedding, and illustrate its use for various problems.  ...  Språk chine learning of substantial current interest are manifold learning and unsupervised and semi-supervised regression.  ...  In [2] , support vector machines is extended to work under the semi-supervised smoothness assumption.  ... 
doi:10.1007/978-3-642-11278-2_23 fatcat:o7xggcrnrzf7lmt2qun4qzp7ky

Selecting Optimal Decisions via Distributionally Robust Nearest-Neighbor Regression

Ruidi Chen, Ioannis Ch. Paschalidis
2019 Neural Information Processing Systems  
A clinically meaningful threshold level used to activate the randomized policy is also derived under a sub-Gaussian assumption on the predicted outcome.  ...  Neighbors (K-NN) regression, which helps to capture the nonlinearity embedded in the data.  ...  Acknowledgments The research was partially supported by the NSF under grants IIS-1914792, DMS-1664644, and CNS-1645681, by the ONR under grant N00014-19-1-2571, by the NIH under grant 1R01GM135930, by  ... 
dblp:conf/nips/ChenP19 fatcat:oftdwl74evcsfnkpxm453bzbd4

Multi-class classification in nonparametric active learning

Boris Ndjia Njike, Xavier Siebert
2022 International Conference on Artificial Intelligence and Statistics  
Several works have recently focused on nonparametric active learning, especially in the binary classification setting under Hölder smoothness assumptions on the regression function.  ...  We present a new algorithm called MKAL for multiclass k-nearest neighbors active learning, and prove its theoretical benefits.  ...  Journal of Machine Learning Research, 22(151):1-25, 2021. Samory Kpotufe. k-nn regression adapts to local intrinsic dimension.  ... 
dblp:conf/aistats/NjikeS22 fatcat:bnqhnuflereibk7vjz3xkpuv6i

A neural network for recovering 3D shape from erroneous and few depth maps of shaded images

Mohamad Ivan Fanany, Itsuo Kumazawa
2004 Pattern Recognition Letters  
Through this analytic mapping, the NN can analytically refine vertices position of the model using error back-propagation learning.  ...  In this paper, we present a new neural network (NN) for three-dimensional (3D) shape reconstruction.  ...  This learning is performed by iteratively compute the gradient descent as follows: v m k ¼ v mÀ1 k À g oE ov k ðk ¼ 0; 1; . . . ; K À 1Þ; ð9Þ where g is learning rate constant and K is total number of  ... 
doi:10.1016/j.patrec.2003.11.001 fatcat:ahthvdfgvzfl3lfueegncis36i

CoNSoLe: Convex Neural Symbolic Learning [article]

Haoran Li, Yang Weng, Hanghang Tong
2022 arXiv   pre-print
In this paper, we propose Convex Neural Symbolic Learning (CoNSoLe) to seek convexity under mild conditions.  ...  Recent advances rely on Neural Networks (NNs) but do not provide theoretical guarantees in obtaining the exact equations owing to the non-convexity of NNs.  ...  Moreover, we quantify the local regions and show the range of the region is large under mild assumptions.  ... 
arXiv:2206.00257v1 fatcat:nkei35e7mjftbjb6lruh4wavoi

Distributed learning algorithm for non-linear differential graphical games

Farzaneh Tatari, Mohammad-Bagher Naghibi-Sistani, Kyriakos G. Vamvoudakis
2016 Transactions of the Institute of Measurement and Control  
The error dynamics and the user-defined performance indices of each agent depend only on local information and the proposed cooperative learning algorithm learns the solution to the cooperative coupled  ...  In the proposed algorithm, each one of the agents uses an actor/critic neural network (NN) structure with appropriate tuning laws in order to guarantee closed-loop stability and convergence of the policies  ...  Assumption 1 contains standard assumptions in NN literature Lewis, 2010, 2011; Zhang et al., 2012a) . Assumption 1(b) is satisfied, e.g. by sigmoids, tanh and other standard NN activation functions.  ... 
doi:10.1177/0142331215603791 fatcat:xtfioyr5s5bzdaf6dc7bww2sjm

On the Complexity of Learning Neural Networks [article]

Le Song, Santosh Vempala, John Wilmes, Bo Xie
2017 arXiv   pre-print
A first step might be to show that data generated by neural networks with a single hidden layer, smooth activation functions and benign input distributions can be learned efficiently.  ...  Moreover, this hard family of functions is realizable with a small (sublinear in dimension) number of activation units in the single hidden layer.  ...  assumptions might make learning NNs tractable.  ... 
arXiv:1707.04615v1 fatcat:bhlgduclevh57gmjpxx46krw6a

Supervised neighborhood graph construction for semi-supervised classification

Mohammad Hossein Rohban, Hamid R. Rabiee
2012 Pattern Recognition  
Graph based methods are among the most active and applicable approaches studied in semi-supervised learning.  ...  Therefore, we propose to use all the subgraphs of k 0 -NNs graph as the hypothesis space.  ...  We have shown that under the using of large enough manifold sampling rate, the optimal neighborhood graph is subgraph of a k 0 -NN graph with high probability.  ... 
doi:10.1016/j.patcog.2011.09.001 fatcat:zgtei6ygzfcjfdmplzencidqp4
« Previous Showing results 1 — 15 out of 11,713 results