Filters








372 Hits in 8.5 sec

Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability

Jinyu Li, Rui Zhao, Zhong Meng, Yanqing Liu, Wenning Wei, Sarangarajan Parthasarathy, Vadim Mazalov, Zhenghao Wang, Lei He, Sheng Zhao, Yifan Gong
2020 Interspeech 2020  
When trained with Microsoft's 65 thousand hours of anonymized training data, the developed RNN-T model surpasses a very well trained hybrid model with both better recognition accuracy and lower latency  ...  In this paper, we describe our recent development of RNN-T models with reduced GPU memory consumption during training, better initialization strategy, and advanced encoder modeling with future lookahead  ...  Different from the 2-pass E2E system, this study focuses on developing a single RNN-T model surpassing a highperformance hybrid model [17] which was developed by integrating 3-stage training and advanced  ... 
doi:10.21437/interspeech.2020-3016 dblp:conf/interspeech/LiZMLWPMWHZG20 fatcat:ife5wu5aijhpbdjsnce4ktt5gy

Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability [article]

Jinyu Li, Rui Zhao, Zhong Meng, Yanqing Liu, Wenning Wei, Sarangarajan Parthasarathy, Vadim Mazalov, Zhenghao Wang, Lei He, Sheng Zhao, Yifan Gong
2020 arXiv   pre-print
When trained with Microsoft's 65 thousand hours of anonymized training data, the developed RNN-T model surpasses a very well trained hybrid model with both better recognition accuracy and lower latency  ...  In this paper, we describe our recent development of RNN-T models with reduced GPU memory consumption during training, better initialization strategy, and advanced encoder modeling with future lookahead  ...  Different from the 2-pass E2E system, this study focuses on developing a single RNN-T model surpassing a highperformance hybrid model [17] which was developed by integrating 3-stage training and advanced  ... 
arXiv:2007.15188v1 fatcat:tkr3z4ryw5h6jpsba6xfxsmyi4

Recent Advances in End-to-End Automatic Speech Recognition [article]

Jinyu Li
2022 arXiv   pre-print
Recently, the speech community is seeing a significant trend of moving from deep neural network based hybrid modeling to end-to-end (E2E) modeling for automatic speech recognition (ASR).  ...  While E2E models achieve the state-of-the-art results in most benchmarks in terms of ASR accuracy, hybrid models are still used in a large proportion of commercial ASR systems at the current time.  ...  With the natural streaming capability, RNN-T becomes the most popular E2E model in the industry [9, 11, 12, 17, [65] [66] [67] [68] [69] .  ... 
arXiv:2111.01690v2 fatcat:6pktwep34jdvjklw4gkri4yn4y

Aspect based sentiment analysis for demonetization tweets by optimized recurrent neural network using fire fly-oriented multi-verse optimizer

Samik Datta, Satyajit Chakrabarti
2021 Sadhana (Bangalore)  
As a modification to the existing RNN, the hidden neurons are optimized by the hybrid FF-MVO, FF-MVO-RNN classifies the positive and negative sentiments.  ...  This pre-processed data is further performed with aspect extraction to extract the opinion words.  ...  FF-MVO-RNN has high performance when compared to the other classifiers. The reasons for the high performance of the proposed classifier is that it uses the RNN, which posses several merits.  ... 
doi:10.1007/s12046-021-01608-1 fatcat:hbqe3oc6evee3bie4jspdoks6a

Deep Learning-Guided Production Quality Estimation for Virtual Environment-Based Applications

2020 Tehnički Vjesnik  
The proposed customized LSTM model with custom batch-wise SMOTE + ENN achieved 99.9% accuracy with an f1 score of 95%.  ...  We also propose a customized LSTM model that is trained to ensure high accuracy in the quality estimation system. This is achieved by our proposed batch-wise balanced training method.  ...  LSTM is a specialized type of RNN with a memory cell, making it capable of learning from very long sequences [19] .  ... 
doi:10.17559/tv-20200906191853 fatcat:xmeraygxibeohbatgu5irfsw2i

Deep Convolutional Neural Network with RNNs for Complex Activity Recognition Using Wrist-Worn Wearable Sensor Data

Sakorn Mekruksavanich, Anuchit Jitpattanakul
2021 Electronics  
Experimental studies on the UTwente dataset demonstrated that the suggested hybrid RNN-based models achieved a high level of recognition performance along with a variety of performance indicators, including  ...  The efficiency of four hybrid DL models that combine convolutional layers with the efficient RNN-based models was also studied.  ...  Moreover, these hybrid models were also employed to distinguish complex activity with high accuracy.  ... 
doi:10.3390/electronics10141685 fatcat:zvruy7iqkfbvvll4eml7cw2lca

Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going [article]

Erwei Wang, James J. Davis, Ruizhe Zhao, Ho-Cheung Ng, Xinyu Niu, Wayne Luk, Peter Y. K. Cheung, George A. Constantinides
2019 arXiv   pre-print
In this article, we provide a comprehensive evaluation of approximation methods for high-performance network inference along with in-depth discussion of their effectiveness for custom hardware implementation  ...  Research has shown that custom hardware-based neural network accelerators can surpass their general-purpose processor equivalents in terms of both throughput and energy efficiency.  ...  Such so ware allows DNN architects unfamiliar with hardware development to migrate their designs to custom hardware with relative ease.  ... 
arXiv:1901.06955v3 fatcat:rkgo2oisdrgv3dtnbtlldlkpba

Deep Learning in Smart Grid Technology: A Review of Recent Advancements and Future Prospects

Mohamed Massaoudi, Haitham Abu-Rub, Shady S. Refaat, Ines Chihi, Fakhreddine S. Oueslati
2021 IEEE Access  
We can see that CNN and RNN models have a high applicability and universality potential rather than the newly developed models. Paper [111] proposed a hybrid method for wind energy forecasting.  ...  In [114] , a cybersecurity diagnosis and localization method using hybridization of AE, RNN, LSTM, and DNN has been proposed.  ...  Abu-Rub has been with Texas A&M University at Qatar.  ... 
doi:10.1109/access.2021.3071269 fatcat:77gyjqaj2zeznc57r4n7kfbu7e

Deep code comment generation with hybrid lexical and syntactical information

Xing Hu, Ge Li, Xin Xia, David Lo, Zhi Jin
2019 Empirical Software Engineering  
Developers have to infer the functionality from the source code.  ...  During software maintenance, developers spend a lot of time understanding the source code.  ...  Investigating the Comments with High BLEU Score There are many comments generated by Hybrid-DeepCom with high BLEU score and we are interested in which kinds of comments with high BLEU score.  ... 
doi:10.1007/s10664-019-09730-9 fatcat:hkeqfnlrfneo7dojiwhagpvtfm

A Hybrid Intelligent Framework to Combat Sophisticated Threats in Secure Industries

Danish Javeed, Tianhan Gao, Muhammad Taimoor Khan, Duaa Shoukat
2022 Sensors  
We have further compared the proposed model with other existing standard classifiers for a thorough performance evaluation.  ...  A hybrid classifier is used for threat detection purposes, i.e., Cu-LSTMGRU + Cu-BLSTM. The proposed model achieved a better detection accuracy with low false-positive rate.  ...  Conflicts of Interest: The authors declare no conflicts of interest associated with this research work.  ... 
doi:10.3390/s22041582 pmid:35214481 pmcid:PMC8875738 fatcat:7rsu74m6inbybp32c2dm7vk4oq

Framework for Deep Learning-Based Language Models using Multi-task Learning in Natural Language Understanding: A Systematic Literature Review and Future Directions

Rahul Manohar Samant, Mrinal Bachute, Shilpa Gite, Ketan Kotecha
2022 IEEE Access  
Unfortunately, these models cannot be generalized for all the NLP tasks with similar performance.  ...  This SLR points out that the unsupervised learning methodbased language models show potential performance improvement.  ...  MODELS WITH HYBRID TECHNIQUES Many hybrid models have been built to detect global and local documents by combining LSTM and CNN architectures.  ... 
doi:10.1109/access.2022.3149798 fatcat:k3kdt4eryzdfpk5k6w62jtlskm

AI-Assisted Framework for Green-Routing and Load Balancing in Hybrid Software-Defined Networking: Proposal, Challenges and Future Perspective

Richard Etengu, Saw Chin Tan, Ching Kwang Lee, Fouad Mohammed Abbou, Teong Che Chuah
2020 IEEE Access  
Second, the recent progress in ML and DRL techniques have proved to surpass human level performance in addressing extensive online control tasks.  ...  Based on the need to minimize global network energy consumption and improve link performance, this paper provides key research insights into the current progress in hybrid SDN/OSPF, ML and AI in the hope  ...  To develop the prediction model, we perform two principle processes, that include hyperparameter tuning and RNN Model Training and Testing.  ... 
doi:10.1109/access.2020.3022291 fatcat:ebznmgl4gfde3kbrewuiphgtme

HARC-New Hybrid Method with Hierarchical Attention Based Bidirectional Recurrent Neural Network with Dilated Convolutional Neural Network to Recognize Multilabel Emotions from Text

Md Shofiqul Islam, Mst Sunjida Sultana, Mr Uttam Kumar, Jubayer Al Mahmud, SM Jahidul Islam
2021 Jurnal Ilmiah Teknik Elektro Komputer dan Informatika  
Our method works better than other basic and CNN and RNN based hybrid models. In the future, we will work for more levels of text emotions from long and more complex text.  ...  performance.  ...  Our proposed attention-based hybrid neural network model is capable of effectively obtaining implicit semantic knowledge.  ... 
doi:10.26555/jiteki.v7i1.20550 fatcat:xnes4y6p2fawfcfiuxecfcu2ru

A Data-Driven Model to Forecast Multi-Step Ahead Time Series of Turkish Daily Electricity Load

Kamil Demirberk Ünlü
2022 Electronics  
This study offers a deep learning methodology to model and forecast multistep daily Turkish electricity loads using the data between 5 January 2015, and 26 December 2021.  ...  Three different performance metrics including coefficient of determination (R2), root mean squared error, and mean absolute error were used to evaluate the performance of the proposed algorithms.  ...  A new feature selection algorithm with a hybrid deep learning methodology based on Elman neural network (ENN) and ridgelet neural network (rNN) was suggested by [18] to model electricity loads of Australia  ... 
doi:10.3390/electronics11101524 fatcat:e2u6uoaonrat3cfmzmf7c655ji

BiGRU-CNN Neural Network Applied to Electric Energy Theft Detection

Lucas Duarte Soares, Altamira de Souza Queiroz, Gloria P. López, Edgar M. Carreño-Franco, Jesús M. López-Lezama, Nicolás Muñoz-Galeano
2022 Electronics  
The use of such a tool with this classification model can help energy sector companies to make decisions regarding theft detection.  ...  The best detection model was that of the BiGRU-CNN artificial neural network when compared to multilayer perceptron, recurrent neural network, gated recurrent unit, and long short-term memory networks.  ...  Recurring neural networks of GRU architecture can be used with other architectures to form hybrid models of electric power fraud detection.  ... 
doi:10.3390/electronics11050693 fatcat:rbiwiukdanat7nruhrr5qhhxgi
« Previous Showing results 1 — 15 out of 372 results