Filters








11 Hits in 1.4 sec

NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing [article]

Nikita Klyuchnikov, Ilya Trofimov, Ekaterina Artemova, Mikhail Salnikov, Maxim Fedorov, Evgeny Burnaev
2020 arXiv   pre-print
Neural Architecture Search (NAS) is a promising and rapidly evolving research area. Training a large number of neural networks requires an exceptional amount of computational power, which makes NAS unreachable for those researchers who have limited or no access to high-performance clusters and supercomputers. A few benchmarks with precomputed neural architectures performances have been recently introduced to overcome this problem and ensure more reproducible experiments. However, these
more » ... s are only for the computer vision domain and, thus, are built from the image datasets and convolution-derived architectures. In this work, we step outside the computer vision domain by leveraging the language modeling task, which is the core of natural language processing (NLP). Our main contribution is as follows: we have provided search space of recurrent neural networks on the text datasets and trained 14k architectures within it; we have conducted both intrinsic and extrinsic evaluation of the trained models using datasets for semantic relatedness and language understanding evaluation; finally, we have tested several NAS algorithms to demonstrate how the precomputed results can be utilized. We believe that our results have high potential of usage for both NAS and NLP communities.
arXiv:2006.07116v1 fatcat:ct5ibdzgevajbc5hy6rvlpflrq

A Differentiable Language Model Adversarial Attack on Text Classifiers [article]

Ivan Fursov, Alexey Zaytsev, Pavel Burnyshev, Ekaterina Dmitrieva, Nikita Klyuchnikov, Andrey Kravchenko, Ekaterina Artemova, Evgeny Burnaev
2021 arXiv   pre-print
Robustness of huge Transformer-based models for natural language processing is an important issue due to their capabilities and wide adoption. One way to understand and improve robustness of these models is an exploration of an adversarial attack scenario: check if a small perturbation of an input can fool a model. Due to the discrete nature of textual data, gradient-based adversarial methods, widely used in computer vision, are not applicable per~se. The standard strategy to overcome this
more » ... is to develop token-level transformations, which do not take the whole sentence into account. In this paper, we propose a new black-box sentence-level attack. Our method fine-tunes a pre-trained language model to generate adversarial examples. A proposed differentiable loss function depends on a substitute classifier score and an approximate edit distance computed via a deep learning model. We show that the proposed attack outperforms competitors on a diverse set of NLP problems for both computed metrics and human evaluation. Moreover, due to the usage of the fine-tuned language model, the generated adversarial examples are hard to detect, thus current models are not robust. Hence, it is difficult to defend from the proposed attack, which is not the case for other attacks.
arXiv:2107.11275v1 fatcat:ava6j4azlzagfm2ibbcks32n7q

Multi-fidelity Neural Architecture Search with Knowledge Distillation [article]

Ilya Trofimov, Nikita Klyuchnikov, Mikhail Salnikov, Alexander Filippov, Evgeny Burnaev
2021 arXiv   pre-print
Neural architecture search (NAS) targets at finding the optimal architecture of a neural network for a problem or a family of problems. Evaluations of neural architectures are very time-consuming. One of the possible ways to mitigate this issue is to use low-fidelity evaluations, namely training on a part of a dataset, fewer epochs, with fewer channels, etc. In this paper, we propose a bayesian multi-fidelity method for neural architecture search: MF-KD. The method relies on a new approach to
more » ... w-fidelity evaluations of neural architectures by training for a few epochs using a knowledge distillation. Knowledge distillation adds to a loss function a term forcing a network to mimic some teacher network. We carry out experiments on CIFAR-10, CIFAR-100, and ImageNet-16-120. We show that training for a few epochs with such a modified loss function leads to a better selection of neural architectures than training for a few epochs with a logistic loss. The proposed method outperforms several state-of-the-art baselines.
arXiv:2006.08341v2 fatcat:acpborr4avejtnlswd7k5adlc4

Gaussian Process Classification for Variable Fidelity Data [article]

Nikita Klyuchnikov, Evgeny Burnaev
2019 arXiv   pre-print
In this paper we address a classification problem where two sources of labels with different levels of fidelity are available. Our approach is to combine data from both sources by applying a co-kriging schema on latent functions, which allows the model to account item-dependent labeling discrepancy. We provide an extension of Laplace inference for Gaussian process classification, that takes into account multi-fidelity data. We evaluate the proposed method on real and synthetic datasets and show
more » ... that it is more resistant to different levels of discrepancy between sources than other approaches for data fusion. Our method can provide accuracy/cost trade-off for a number of practical tasks such as crowd-sourced data annotation and feasibility regions construction in engineering design.
arXiv:1809.05143v3 fatcat:hveqcpi3evcdblfjxaumfbeeie

Diverse genetic origins of medieval steppe nomad conquerors [article]

Alexander Mikheyev, Lijun Qiu, Alexei Zarubin, Nikita Moshkov, Yuri Orlov, Duane Chartier, Tatiana Faleeva, Igor Kornienko, Vladimir Klyuchnikov, Elena Batieva, Tatiana V Tatarinova
2019 bioRxiv   pre-print
Over millennia, steppe nomadic tribes raided and sometimes overran settled Eurasian civilizations. Most polities formed by steppe nomads were ephemeral, making it difficult to ascertain their genetic roots or what present-day populations, if any, have descended from them. Exceptionally, the Khazar Khaganate controlled the trade artery between the Black and Caspian Seas in VIII-IX centuries, acting as one of the major conduits between East and West. However, the genetic identity of the ruling
more » ... te within the polyglot and polyethnic Khaganate has been a much-debated mystery; a controversial hypothesis posits that post-conversion to Judaism the Khazars gave rise to modern Ashkenazim. We analyzed whole-genome sequences of eight men and one woman buried within the distinctive kurgans of the Khazar upper (warrior) class. After comparing them with reference panels of present-day Eurasian and Iron Age populations, we found that the Khazar political organization relied on a polyethnic elite. It was predominantly descended from Central Asian tribes but incorporated genetic admixture from populations conquered by Khazars. Thus, the Khazar ruling class was likely relatively small and able to maintain a genetic identity distinct from their subjugated populations over the course of centuries. Yet, men of mixed ancestry could also rise into the warrior class, possibly providing troop numbers necessary to maintain control of their large territory. However, when the Khaganate collapsed it left few persistent genetic traces in Europe. Our data confirm the Turkic roots of the Khazars, but also highlight their ethnic diversity and some integration of conquered populations.
doi:10.1101/2019.12.15.876912 fatcat:p7yumgwrvvhotjcywf5hvduwiq

Application of Machine Learning to accidents detection at directional drilling [article]

Ekaterina Gurina, Nikita Klyuchnikov, Alexey Zaytsev, Evgenya Romanenkova, Ksenia Antipova, Igor Simon, Victor Makarov, Dmitry Koroteev
2019 arXiv   pre-print
., 2016) , lithology classification (Klyuchnikov et al., 2018; Romanenkova et al., 2019) ; in upstream, for example, engineers usually use it for detection sensors faults in a refinery (Saybani et al  ... 
arXiv:1906.02667v2 fatcat:qgpmwtjmgfh73fs3ibfadwmle4

Forecasting the abnormal events at well drilling with machine learning

Ekaterina Gurina, Nikita Klyuchnikov, Ksenia Antipova, Dmitry Koroteev
2022 Applied intelligence (Boston)  
We present a data-driven and physics-informed algorithm for drilling accident forecasting. The core machine-learning algorithm uses the data from the drilling telemetry representing the time-series. We have developed a Bag-of-features representation of the time series that enables the algorithm to predict the probabilities of six types of drilling accidents in real-time. The machine-learning model is trained on the 125 past drilling accidents from 100 different Russian oil and gas wells.
more » ... ion shows that the model can forecast 70% of drilling accidents with a false positive rate equals to 40%. The model addresses partial prevention of the drilling accidents at the well construction.
doi:10.1007/s10489-021-03013-x fatcat:xegne323kzbw7gggdm2x2s56pu

Real-time data-driven detection of the rock type alteration during a directional drilling [article]

Evgenya Romanenkova, Alexey Zaytsev, Nikita Klyuchnikov, Arseniy Gruzdev, Ksenia Antipova, Leyla Ismailova, Evgeny Burnaev, Artyom Semenikhin, Vitaliy Koryabkin, Igor Simon, Dmitry Koroteev
2019 arXiv   pre-print
Klyuchnikov To solve the problem at hand, we train two data-driven models: a machine learning (ML) classifier and a threshold-based change point detection algorithm.  ... 
arXiv:1903.11436v2 fatcat:4c5j6u3ixzg73n77mla56zi7ia

Data-driven model for the identification of the rock type at a drilling bit [article]

Nikita Klyuchnikov, Alexey Zaytsev, Arseniy Gruzdev, Georgiy Ovchinnikov, Ksenia Antipova, Leyla Ismailova, Ekaterina Muravleva, Evgeny Burnaev, Artyom Semenikhin, Alexey Cherepanov, Vitaliy Koryabkin, Igor Simon, Alexey Tsurgan (+2 others)
2019 arXiv   pre-print
Directional oil well drilling requires high precision of the wellbore positioning inside the productive area. However, due to specifics of engineering design, sensors that explicitly determine the type of the drilled rock are located farther than 15m from the drilling bit. As a result, the target area runaways can be detected only after this distance, which in turn, leads to a loss in well productivity and the risk of the need for an expensive re-boring operation. We present a novel approach
more » ... identifying rock type at the drilling bit based on machine learning classification methods and data mining on sensors readings. We compare various machine-learning algorithms, examine extra features coming from mathematical modeling of drilling mechanics, and show that the real-time rock type classification error can be reduced from 13.5 % to 9 %. The approach is applicable for precise directional drilling in relatively thin target intervals of complex shapes and generalizes appropriately to new wells that are different from the ones used for training the machine learning model.
arXiv:1806.03218v3 fatcat:tqnttizqkfbxxfjcsfvkylivmy

МОДЕЛИРОВАНИЕ ВСПОМОГАТЕЛЬНЫХ МЕР ПО СТАБИЛИЗАЦИИ ЦЕНОВОЙ ДИНАМИКИ НА РЫНКЕ НЕФТИ

Александр Иванович Звягинцев
2019 Современная экономика проблемы и решения  
Klimov Nikita Aleksandrovich, student voronezh State University, University sq., 1, voronezh, Russia, 394018; e-mail: gmasha3@gmail.com; treschevsky@gmail.com; klimnik1999@mail.ru .  ...  ., Klyuchnikov I.K. Mekha nizm ekonomicheskogo rosta trans natsi onalynykh korporatsiy [The economic growth mechanism of transnational corpo rations]. Moscow, vyssh. shk., 1990. (In Russ.) 6.  ... 
doi:10.17308/meps.2019.6/2131 fatcat:onza2a73jfgjpm22oplsdzmlxa

Post-Soviet 'Uncivil Society' and the Rise of Aleksandr Dugin: A Case Study of the Extraparliamentary Radical Right in Contemporary Russia

Andreas Umland
2007 Social Science Research Network  
Klyuchnikov, "Russkii uzel evraziistva," Nash sovremennik, no. 3 (1992): 174-180.  ...  See Nikita Kaledin, "Terapiya okazalas' bessil'noi pered maniei Dugina-mladshego elaborations on Dugin's ideas go only as far as is necessary to illustrate what his political strategy is about, and, in  ... 
doi:10.2139/ssrn.2892325 fatcat:bki4p5tcivdevdgy4yz7mxq5oa