3,326 Hits in 3.4 sec

Natural Compression for Distributed Deep Learning [article]

Samuel Horvath, Chen-Yu Ho, Ludovit Horvath, Atal Narayan Sahu, Marco Canini, Peter Richtarik
2020 arXiv   pre-print
Correspondence to: Samuel Horvath <>. Under review. and dimension and complexity of the models required to obtain state-of-the-art performance.  ...  Horváth, S. and Richtárik, P. Nonconvex variance reduced optimization with arbitrary sampling. In Chaudhuri, K. and Salakhutdinov, R.  ... 
arXiv:1905.10988v2 fatcat:cwk3l74cfjag5nt6v6nqlleuci

Hyperparameter Transfer Learning with Adaptive Complexity [article]

Samuel Horváth, Aaron Klein, Peter Richtárik, Cédric Archambeau
2021 arXiv   pre-print
Bayesian optimization (BO) is a sample efficient approach to automatically tune the hyperparameters of machine learning models. In practice, one frequently has to solve similar hyperparameter tuning problems sequentially. For example, one might have to tune a type of neural network learned across a series of different classification problems. Recent work on multi-task BO exploits knowledge gained from previous tuning tasks to speed up a new tuning task. However, previous approaches do not
more » ... t for the fact that BO is a sequential decision making procedure. Hence, there is in general a mismatch between the number of evaluations collected in the current tuning task compared to the number of evaluations accumulated in all previously completed tasks. In this work, we enable multi-task BO to compensate for this mismatch, such that the transfer learning procedure is able to handle different data regimes in a principled way. We propose a new multi-task BO method that learns a set of ordered, non-linear basis functions of increasing complexity via nested drop-out and automatic relevance determination. Experiments on a variety of hyperparameter tuning problems show that our method improves the sample ef
arXiv:2102.12810v1 fatcat:nlj6s6i4mvh6villxindirb24m

Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization [article]

Samuel Horváth, Lihua Lei, Peter Richtárik, Michael I. Jordan
2020 arXiv   pre-print
Adaptivity is an important yet under-studied property in modern optimization theory. The gap between the state-of-the-art theory and the current practice is striking in that algorithms with desirable theoretical guarantees typically involve drastically different settings of hyperparameters, such as step-size schemes and batch sizes, in different regimes. Despite the appealing theoretical results, such divisive strategies provide little, if any, insight to practitioners to select algorithms that
more » ... work broadly without tweaking the hyperparameters. In this work, blending the "geometrization" technique introduced by Lei Jordan 2016 and the SARAH algorithm of Nguyen et al., 2017, we propose the Geometrized SARAH algorithm for non-convex finite-sum and stochastic optimization. Our algorithm is proved to achieve adaptivity to both the magnitude of the target accuracy and the Polyak-Łojasiewicz (PL) constant if present. In addition, it achieves the best-available convergence rate for non-PL objectives simultaneously while outperforming existing algorithms for PL objectives.
arXiv:2002.05359v1 fatcat:34r5nlw4krg4xe2vhlxe4axdwu

Optimal Client Sampling for Federated Learning [article]

Wenlin Chen, Samuel Horvath, Peter Richtarik
2021 arXiv   pre-print
In order to provide an analysis in this framework, we consider a general partial participation framework [Horváth and Richtárik, 2020] , where we assume that the subset of participating clients is determined  ... 
arXiv:2010.13723v2 fatcat:hlrdoe6idbaajj4jvpzliew6du

Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneity [article]

Samuel Horváth
2022 arXiv   pre-print
This chapter is based on: [53] : Samuel Horváth, Chen-Yu Ho, Ľudovít Horváth, Atal Narayan Sahu, Marco Canini, and Peter Richtárik, "Natural compression for distributed deep learning", arXiv preprint  ...  This chapter is based on: [57]: Wenlin Chen, Samuel Horváth, and Peter Richtárik, "Optimal client sampling for federated learning", arXiv preprint arXiv:2010.13723, 2020.  ... 
arXiv:2207.00392v1 fatcat:4ulwzbpusjbntnijrwe4vlgspe

Lower Bounds and Optimal Algorithms for Personalized Federated Learning [article]

Filip Hanzely, Slavomír Hanzely, Samuel Horváth, Peter Richtárik
2020 arXiv   pre-print
In this work, we consider the optimization formulation of personalized federated learning recently introduced by Hanzely and Richtárik (2020) which was shown to give an alternative explanation to the workings of local SGD methods. Our first contribution is establishing the first lower bounds for this formulation, for both the communication complexity and the local oracle complexity. Our second contribution is the design of several optimal methods matching these lower bounds in almost all
more » ... . These are the first provably optimal methods for personalized federated learning. Our optimal methods include an accelerated variant of FedProx, and an accelerated variance-reduced version of FedAvg/Local SGD. We demonstrate the practical superiority of our methods through extensive numerical experiments.
arXiv:2010.02372v1 fatcat:botvsoawe5fobf64dj4frjoc2e

Nonconvex Variance Reduced Optimization with Arbitrary Sampling [article]

Samuel Horváth, Peter Richtárik
2019 arXiv   pre-print
We provide the first importance sampling variants of variance reduced algorithms for empirical risk minimization with non-convex loss functions. In particular, we analyze non-convex versions of SVRG, SAGA and SARAH. Our methods have the capacity to speed up the training process by an order of magnitude compared to the state of the art on real datasets. Moreover, we also improve upon current mini-batch analysis of these methods by proposing importance sampling for minibatches in this setting.
more » ... prisingly, our approach can in some regimes lead to superlinear speedup with respect to the minibatch size, which is not usually present in stochastic optimization. All the above results follow from a general analysis of the methods which works with arbitrary sampling, i.e., fully general randomized strategy for the selection of subsets of examples to be sampled in each iteration. Finally, we also perform a novel importance sampling analysis of SARAH in the convex setting.
arXiv:1809.04146v2 fatcat:vajcltes5nenzbhd23fjsswuem

On Biased Compression for Distributed Learning [article]

Aleksandr Beznosikov and Samuel Horváth and Peter Richtárik and Mher Safaryan
2021 arXiv   pre-print
Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Sebastian Stich, and Peter Richtárik.  ...  Samuel Horváth, Chen-Yu Ho, Ľudovít Horváth, Atal Narayan Sahu, Marco Canini, and Peter Richtárik. Natural compression for distributed deep learning. arXiv preprint arXiv:1905.10988, 2019a.  ... 
arXiv:2002.12410v2 fatcat:n45gzaj5z5fy5mbwa5ux66mxku

Experimental investigation of the influence of temperature on thermal conductivity of multilayer reflective thermal insulation

Zoltán Pásztory, Tibor Horváth, Samuel V. Glass, Samuel Zelinka
2018 Energy and Buildings  
The apparent thermal conductivity of several insulation materials was measured over a range of temperatures. A newly developed multilayer reflective insulation system called Mirrorpanel was tested against existing products. Mirrorpanel samples were prepared using layers of aluminum foil (emissivity of 0.11) and coated paper (emissivity of 0.52) separated by air spaces of approximately 5 mm, with fiberboard structural spacers. Steady-state heat flux was measured in the laboratory for 500 mm ×
more » ... mm samples including several Mirrorpanel configurations as well as expanded polystyrene and polyisocyanurate foam insulations. The mean temperature ranged between 0 °C and 35 °C with a temperature difference across the sample of 10 °C. For all insulation materials, the apparent thermal conductivity increased linearly with temperature above 5 °C, and the slope was steeper for the Mirrorpanel samples than the foam insulations. The apparent thermal conductivity of the Mirrorpanel made from aluminum foil was greater than that of polyisocyanurate but less than that of expanded polystyrene. The significant difference of thermal conductivity of lower and higher emissivity reflecting layers highlighted the importance of this parameter in thermal insulation. The steep temperature dependence of the Mirrorpanel should be considered during design of the building envelope for summer and winter conditions.
doi:10.1016/j.enbuild.2018.06.012 fatcat:bgzhnw77j5gepkwcadhsdtpwiu

Stochastic Distributed Learning with Gradient Quantization and Variance Reduction [article]

Samuel Horváth and Dmitry Kovalev and Konstantin Mishchenko and Sebastian Stich and Peter Richtárik
2019 arXiv   pre-print
., Horváth, S., and Richtárik, P. Don't jump through hoops and remove those loops: Svrg and katyusha are better without the outer loop. arXiv:1901.08689, 2019.Künstner, F.  ... 
arXiv:1904.05115v1 fatcat:leu725ytbzc2zkzlbhu7xlw5pi

Epilepsziasebészeti beavatkozások eredményei a Pécsi Epilepszia Centrumban 2005 és 2016 között

Katalin Nóra Lőrincz, Beáta Bóné, Márton Tóth, Réka Horváth, Norbert Kovács, Sámuel Komoly, Kázmér Karádi, Péter Barsi, Hajnalka Ábrahám, László Seress, Zsolt Horváth, Tamás Dóczi (+2 others)
2019 Orvosi Hetilap  
Köszönetnyilvánítás Horváth Rékát "Az Emberi Erőforrások Minisztériuma ÚNKP-17-4-I-PTE-139 kódszámú Új Nemzeti Kiválóság Programja" támogatta.  ...  Lőrincz KN, Bóné B, Tóth M, Horváth R, Kovács N, Komoly S, Karádi K, Barsi Az epilepszia a stroke után a második leggyakoribb neurológiai betegség, mely Magyarországon megközelítőleg 50-60 ezer embert  ... 
doi:10.1556/650.2019.31321 fatcat:4zhhv4xuobb77mpus52cjedfii

Status epilepticus 2020

József Janszky, Beáta Bóné, Réka Horváth, Zsófia Sütő, László Szapáry, Vera Juhos, Sámuel Komoly, Norbert Kovács
2020 Orvosi Hetilap  
Absztrakt: A status epilepticus a második leggyakoribb, sürgősségi kezelést igénylő neurológiai állapot. Halálozása 15–25%. A "time is brain" elve a status epilepticus kezelésére is igaz: minél korábban kezdjük a megfelelő kezelést, annál nagyobb valószínűséggel tudjuk megállítani a progressziót. Magas szintű evidenciákon alapuló kezelési protokollal a status epilepticus progressziója az esetek 75–90%-ában megelőzhető, az indukált kóma és a halálos kimenetel elkerülhető. A status epilepticus
more » ... elése akkor a legsikeresebb, ha már a korai szakban megkezdjük a parenteralis benzodiazepinterápiát: im. midazolám (0,2 mg/tskg, max. 10 mg). Szabad véna esetén lehet vénásan is adni a benzodiazepint (10 mg diazepám iv). Ha az első benzodiazepinbolusra nem reagál a status epilepticus, állandósult (benzodiazepinrefrakter) status epilepticusról beszélünk. Ilyenkor a benzodiazepin ismétlésével párhuzamosan nem benzodiazepin típusú, gyorsan ható vénás antiepileptikumot is adni kell: iv. valproát (40 mg/kg, max. 3000 mg, 10 perc alatt) vagy levetiracetám (60 mg/kg, max. 4500 mg, 10 perc alatt) javasolt. Az 1 órán túl is tartó, sem benzodiazepinre, sem antiepileptikumra nem reagáló, refrakter status epilepticust neurointenzív osztályon, teljes narcosissal (indukált kómával) kell kezelni. Az indukált kómát gyors hatású anesztetikummal lehet elérni, elsősorban propofol–midazolám kombinációval. Orv Hetil. 2020; 161(42): 1779–1786.
doi:10.1556/650.2020.31908 pmid:33070121 fatcat:ia3ujoj6qbbmxgnn2s7oetjp44

A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning [article]

Samuel Horváth, Peter Richtárik
2021 arXiv   pre-print
., 2017; Horváth et al., 2019a; Ramezani-Kebrya et al., 2019) .  ...  It was shown by Horváth et al. (2019b) that both σ 2 and D can be removed for the setting of Theorem 2.  ... 
arXiv:2006.11077v2 fatcat:rzqmsftyvfguvaetrwuuvbafdy

Binding of Task-Irrelevant Action Features and Auditory Action Effects

Sámuel Varga, Roland Pfister, Bence Neszmélyi, Wilfried Kunde, János Horváth
2022 Journal of Cognition  
., 2020; Horváth et al., 2018; Pfister, 2019) .  ...  Moreover, even though metric properties of the eventual body movement are nominally task-irrelevant by instruction, they are still important when it comes to optimizing movements (Neszmélyi & Horváth,  ... 
doi:10.5334/joc.225 fatcat:v6scifhfszcehlq7u2ozbhhaji

Patient-Driven Findings of Genetic Associations for PANS and PANDAS

Robert Steve Horvath, Samuel Keating
2021 Journal of Biomedicine and Translational Research  
There are presently very few genetic studies for PANS (Pediatric Acute-Onset Neuropsychiatric Syndrome) or PANDAS (Pediatric Autoimmune Neuropsychiatric Disorders Associated with Streptococcal Infections). More work in genetic associations for PANS and PANDAS (P/P) is needed to increase understanding of these debilitating childhood disorders that have a range of presentations.Objective: This work represents a novel approach that aims to determine genetic associations between P/P and other
more » ... es, disorders and traits (hereafter referred to as phenotypes).Methods: Consumer genetic data (23andMe, AncestryDNA) for 155 patients with P/P were obtained from consenting parents over a period from 2018 to 2020. An analysis plan for this work was registered at Open Science Framework, additional genotypes imputed using, and polygenic risk scores for 1,702 phenotypes calculated for each of the 155 P/P patients.Results: One-sample t-tests performed across the 155 individual risk scores revealed that P/P is statistically significantly associated with 21 different groups of Single Nucleotide Polymorphisms (SNPs) that are in turn associated with 21 phenotypes. Some of the 21 phenotypes (see Table 3) are previously known to be related to or associated with P/P: a group of SNPs associated with Tourette's Syndrome, and another group associated with Autism Spectrum Disorder or Schizophrenia, and a third associated with "feeling nervous" yielded t-tests with p values of 1.2x10-5, 1.2x10-11 and 1.0x10-5 respectively for association with the P/P data. This validated our analysis methodology. Our analysis also revealed novel genetic associations such as between P/P and plasma anti-thyroglobulin levels (p=1.3x10-7), between P/P and triglycerides (p=5.6x10-6), and between P/P and Lewy body disease (p=7.8x10-6), inviting further investigation into the underlying etiology of P/P.Conclusion: P/P is associated with many phenotypes not previously recognized as being connected to P/P. Further work on these connections can lead to better understanding of P/P.
doi:10.14710/jbtr.v7i3.12082 fatcat:zxjr54rh4nclfjdmnrrhppa4oy
« Previous Showing results 1 — 15 out of 3,326 results