A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Natural Compression for Distributed Deep Learning
[article]
2020
arXiv
pre-print
Preserved Fulltext
Other Versions
Correspondence to: Samuel Horvath <samuel.horvath@kaust.edu.sa>. Under review. and dimension and complexity of the models required to obtain state-of-the-art performance. ...
Horváth, S. and Richtárik, P. Nonconvex variance reduced
optimization with arbitrary sampling. In Chaudhuri, K.
and Salakhutdinov, R. ...
arXiv:1905.10988v2
fatcat:cwk3l74cfjag5nt6v6nqlleuci
Hyperparameter Transfer Learning with Adaptive Complexity
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Bayesian optimization (BO) is a sample efficient approach to automatically tune the hyperparameters of machine learning models. In practice, one frequently has to solve similar hyperparameter tuning problems sequentially. For example, one might have to tune a type of neural network learned across a series of different classification problems. Recent work on multi-task BO exploits knowledge gained from previous tuning tasks to speed up a new tuning task. However, previous approaches do not
arXiv:2102.12810v1
fatcat:nlj6s6i4mvh6villxindirb24m
more »
... t for the fact that BO is a sequential decision making procedure. Hence, there is in general a mismatch between the number of evaluations collected in the current tuning task compared to the number of evaluations accumulated in all previously completed tasks. In this work, we enable multi-task BO to compensate for this mismatch, such that the transfer learning procedure is able to handle different data regimes in a principled way. We propose a new multi-task BO method that learns a set of ordered, non-linear basis functions of increasing complexity via nested drop-out and automatic relevance determination. Experiments on a variety of hyperparameter tuning problems show that our method improves the sample ef
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Adaptivity is an important yet under-studied property in modern optimization theory. The gap between the state-of-the-art theory and the current practice is striking in that algorithms with desirable theoretical guarantees typically involve drastically different settings of hyperparameters, such as step-size schemes and batch sizes, in different regimes. Despite the appealing theoretical results, such divisive strategies provide little, if any, insight to practitioners to select algorithms that
arXiv:2002.05359v1
fatcat:34r5nlw4krg4xe2vhlxe4axdwu
more »
... work broadly without tweaking the hyperparameters. In this work, blending the "geometrization" technique introduced by Lei Jordan 2016 and the SARAH algorithm of Nguyen et al., 2017, we propose the Geometrized SARAH algorithm for non-convex finite-sum and stochastic optimization. Our algorithm is proved to achieve adaptivity to both the magnitude of the target accuracy and the Polyak-Łojasiewicz (PL) constant if present. In addition, it achieves the best-available convergence rate for non-PL objectives simultaneously while outperforming existing algorithms for PL objectives.
Optimal Client Sampling for Federated Learning
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In order to provide an analysis in this framework, we consider a general partial participation framework [Horváth and Richtárik, 2020] , where we assume that the subset of participating clients is determined ...
arXiv:2010.13723v2
fatcat:hlrdoe6idbaajj4jvpzliew6du
Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneity
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
This chapter is based on: [53] : Samuel Horváth, Chen-Yu Ho, Ľudovít Horváth, Atal Narayan Sahu, Marco Canini, and Peter Richtárik, "Natural compression for distributed deep learning", arXiv preprint ...
This chapter is based on: [57]: Wenlin Chen, Samuel Horváth, and Peter Richtárik, "Optimal client sampling for federated learning", arXiv preprint arXiv:2010.13723, 2020. ...
arXiv:2207.00392v1
fatcat:4ulwzbpusjbntnijrwe4vlgspe
Lower Bounds and Optimal Algorithms for Personalized Federated Learning
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this work, we consider the optimization formulation of personalized federated learning recently introduced by Hanzely and Richtárik (2020) which was shown to give an alternative explanation to the workings of local SGD methods. Our first contribution is establishing the first lower bounds for this formulation, for both the communication complexity and the local oracle complexity. Our second contribution is the design of several optimal methods matching these lower bounds in almost all
arXiv:2010.02372v1
fatcat:botvsoawe5fobf64dj4frjoc2e
more »
... . These are the first provably optimal methods for personalized federated learning. Our optimal methods include an accelerated variant of FedProx, and an accelerated variance-reduced version of FedAvg/Local SGD. We demonstrate the practical superiority of our methods through extensive numerical experiments.
Nonconvex Variance Reduced Optimization with Arbitrary Sampling
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We provide the first importance sampling variants of variance reduced algorithms for empirical risk minimization with non-convex loss functions. In particular, we analyze non-convex versions of SVRG, SAGA and SARAH. Our methods have the capacity to speed up the training process by an order of magnitude compared to the state of the art on real datasets. Moreover, we also improve upon current mini-batch analysis of these methods by proposing importance sampling for minibatches in this setting.
arXiv:1809.04146v2
fatcat:vajcltes5nenzbhd23fjsswuem
more »
... prisingly, our approach can in some regimes lead to superlinear speedup with respect to the minibatch size, which is not usually present in stochastic optimization. All the above results follow from a general analysis of the methods which works with arbitrary sampling, i.e., fully general randomized strategy for the selection of subsets of examples to be sampled in each iteration. Finally, we also perform a novel importance sampling analysis of SARAH in the convex setting.
On Biased Compression for Distributed Learning
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Sebastian Stich, and Peter Richtárik. ...
Samuel Horváth, Chen-Yu Ho, Ľudovít Horváth, Atal Narayan Sahu, Marco Canini, and Peter Richtárik.
Natural compression for distributed deep learning. arXiv preprint arXiv:1905.10988, 2019a. ...
arXiv:2002.12410v2
fatcat:n45gzaj5z5fy5mbwa5ux66mxku
Experimental investigation of the influence of temperature on thermal conductivity of multilayer reflective thermal insulation
2018
Energy and Buildings
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The apparent thermal conductivity of several insulation materials was measured over a range of temperatures. A newly developed multilayer reflective insulation system called Mirrorpanel was tested against existing products. Mirrorpanel samples were prepared using layers of aluminum foil (emissivity of 0.11) and coated paper (emissivity of 0.52) separated by air spaces of approximately 5 mm, with fiberboard structural spacers. Steady-state heat flux was measured in the laboratory for 500 mm ×
doi:10.1016/j.enbuild.2018.06.012
fatcat:bgzhnw77j5gepkwcadhsdtpwiu
more »
... mm samples including several Mirrorpanel configurations as well as expanded polystyrene and polyisocyanurate foam insulations. The mean temperature ranged between 0 °C and 35 °C with a temperature difference across the sample of 10 °C. For all insulation materials, the apparent thermal conductivity increased linearly with temperature above 5 °C, and the slope was steeper for the Mirrorpanel samples than the foam insulations. The apparent thermal conductivity of the Mirrorpanel made from aluminum foil was greater than that of polyisocyanurate but less than that of expanded polystyrene. The significant difference of thermal conductivity of lower and higher emissivity reflecting layers highlighted the importance of this parameter in thermal insulation. The steep temperature dependence of the Mirrorpanel should be considered during design of the building envelope for summer and winter conditions.
Stochastic Distributed Learning with Gradient Quantization and Variance Reduction
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
., Horváth, S., and Richtárik, P. Don't jump through hoops and remove those loops: Svrg and katyusha are better without the outer loop. arXiv:1901.08689, 2019.Künstner, F. ...
arXiv:1904.05115v1
fatcat:leu725ytbzc2zkzlbhu7xlw5pi
Epilepsziasebészeti beavatkozások eredményei a Pécsi Epilepszia Centrumban 2005 és 2016 között
2019
Orvosi Hetilap
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Köszönetnyilvánítás Horváth Rékát "Az Emberi Erőforrások Minisztériuma ÚNKP-17-4-I-PTE-139 kódszámú Új Nemzeti Kiválóság Programja" támogatta. ...
Lőrincz KN, Bóné B, Tóth M, Horváth R, Kovács N, Komoly S, Karádi K, Barsi Az epilepszia a stroke után a második leggyakoribb neurológiai betegség, mely Magyarországon megközelítőleg 50-60 ezer embert ...
doi:10.1556/650.2019.31321
fatcat:4zhhv4xuobb77mpus52cjedfii
Status epilepticus 2020
2020
Orvosi Hetilap
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Absztrakt: A status epilepticus a második leggyakoribb, sürgősségi kezelést igénylő neurológiai állapot. Halálozása 15–25%. A "time is brain" elve a status epilepticus kezelésére is igaz: minél korábban kezdjük a megfelelő kezelést, annál nagyobb valószínűséggel tudjuk megállítani a progressziót. Magas szintű evidenciákon alapuló kezelési protokollal a status epilepticus progressziója az esetek 75–90%-ában megelőzhető, az indukált kóma és a halálos kimenetel elkerülhető. A status epilepticus
doi:10.1556/650.2020.31908
pmid:33070121
fatcat:ia3ujoj6qbbmxgnn2s7oetjp44
more »
... elése akkor a legsikeresebb, ha már a korai szakban megkezdjük a parenteralis benzodiazepinterápiát: im. midazolám (0,2 mg/tskg, max. 10 mg). Szabad véna esetén lehet vénásan is adni a benzodiazepint (10 mg diazepám iv). Ha az első benzodiazepinbolusra nem reagál a status epilepticus, állandósult (benzodiazepinrefrakter) status epilepticusról beszélünk. Ilyenkor a benzodiazepin ismétlésével párhuzamosan nem benzodiazepin típusú, gyorsan ható vénás antiepileptikumot is adni kell: iv. valproát (40 mg/kg, max. 3000 mg, 10 perc alatt) vagy levetiracetám (60 mg/kg, max. 4500 mg, 10 perc alatt) javasolt. Az 1 órán túl is tartó, sem benzodiazepinre, sem antiepileptikumra nem reagáló, refrakter status epilepticust neurointenzív osztályon, teljes narcosissal (indukált kómával) kell kezelni. Az indukált kómát gyors hatású anesztetikummal lehet elérni, elsősorban propofol–midazolám kombinációval. Orv Hetil. 2020; 161(42): 1779–1786.
A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
., 2017; Horváth et al., 2019a; Ramezani-Kebrya et al., 2019) . ...
It was shown by Horváth et al. (2019b) that both σ 2 and D can be removed for the setting of Theorem 2. ...
arXiv:2006.11077v2
fatcat:rzqmsftyvfguvaetrwuuvbafdy
Binding of Task-Irrelevant Action Features and Auditory Action Effects
2022
Journal of Cognition
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
., 2020; Horváth et al., 2018; Pfister, 2019) . ...
Moreover, even though metric properties of the eventual body movement are nominally task-irrelevant by instruction, they are still important when it comes to optimizing movements (Neszmélyi & Horváth, ...
doi:10.5334/joc.225
fatcat:v6scifhfszcehlq7u2ozbhhaji
Patient-Driven Findings of Genetic Associations for PANS and PANDAS
2021
Journal of Biomedicine and Translational Research
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
There are presently very few genetic studies for PANS (Pediatric Acute-Onset Neuropsychiatric Syndrome) or PANDAS (Pediatric Autoimmune Neuropsychiatric Disorders Associated with Streptococcal Infections). More work in genetic associations for PANS and PANDAS (P/P) is needed to increase understanding of these debilitating childhood disorders that have a range of presentations.Objective: This work represents a novel approach that aims to determine genetic associations between P/P and other
doi:10.14710/jbtr.v7i3.12082
fatcat:zxjr54rh4nclfjdmnrrhppa4oy
more »
... es, disorders and traits (hereafter referred to as phenotypes).Methods: Consumer genetic data (23andMe, AncestryDNA) for 155 patients with P/P were obtained from consenting parents over a period from 2018 to 2020. An analysis plan for this work was registered at Open Science Framework, additional genotypes imputed using Impute.me, and polygenic risk scores for 1,702 phenotypes calculated for each of the 155 P/P patients.Results: One-sample t-tests performed across the 155 individual risk scores revealed that P/P is statistically significantly associated with 21 different groups of Single Nucleotide Polymorphisms (SNPs) that are in turn associated with 21 phenotypes. Some of the 21 phenotypes (see Table 3) are previously known to be related to or associated with P/P: a group of SNPs associated with Tourette's Syndrome, and another group associated with Autism Spectrum Disorder or Schizophrenia, and a third associated with "feeling nervous" yielded t-tests with p values of 1.2x10-5, 1.2x10-11 and 1.0x10-5 respectively for association with the P/P data. This validated our analysis methodology. Our analysis also revealed novel genetic associations such as between P/P and plasma anti-thyroglobulin levels (p=1.3x10-7), between P/P and triglycerides (p=5.6x10-6), and between P/P and Lewy body disease (p=7.8x10-6), inviting further investigation into the underlying etiology of P/P.Conclusion: P/P is associated with many phenotypes not previously recognized as being connected to P/P. Further work on these connections can lead to better understanding of P/P.
« Previous
Showing results 1 — 15 out of 3,326 results