Filters








106 Hits in 8.8 sec

Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition [article]

Kenichi Kumatani, Robert Gmyr, Felipe Cruz Salinas, Linquan Liu, Wei Zuo, Devang Patel, Eric Sun, Yu Shi
2022 arXiv   pre-print
The sparsely-gated Mixture of Experts (MoE) can magnify a network capacity with a little computational complexity.  ...  In this work, we investigate how multi-lingual Automatic Speech Recognition (ASR) networks can be scaled up with a simple routing algorithm in order to achieve better accuracy.  ...  We also would like to thank Ed Lin, Michael Zeng, Xuedong Huang and Yuan Yu for their project support.  ... 
arXiv:2112.05820v3 fatcat:pfhih26qpnagnnmic24sempqcy

Software Architecture for Language Engineering

HAMISH CUNNINGHAM, DONIA SCOTT
2004 Natural Language Engineering  
In order to demonstrate the theory developed in relation to SALE, we present the design, implementation and evaluation of GATE, a General Architecture for Text Engineering, which illustrates in practice  ...  The thesis represents the first discussion of software infrastructure for language computation that covers a large portion of the field.  ...  For example, a translator's workbench is an application 2 . Technologies are those areas of research that contain relatively well-defined bodies of theory, methods  ... 
doi:10.1017/s1351324904003481 fatcat:xzkpj2edozgidfrknmergcyyga

A Roadmap for Big Model [article]

Sha Yuan, Hanyu Zhao, Shuai Zhao, Jiahong Leng, Yangxiao Liang, Xiaozhi Wang, Jifan Yu, Xin Lv, Zhou Shao, Jiaao He, Yankai Lin, Xu Han (+88 others)
2022 arXiv   pre-print
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.  ...  At the end of this paper, we conclude the further development of BMs in a more general view.  ...  Expert Parallelism Mixture-of-expert(MoE) is a newly evolving structure for extremely large models beyond trillion scale.  ... 
arXiv:2203.14101v4 fatcat:rdikzudoezak5b36cf6hhne5u4

Multi-Task Learning with Deep Neural Networks: A Survey [article]

Michael Crawshaw
2020 arXiv   pre-print
In this survey, we give an overview of multi-task learning methods for deep neural networks, with the aim of summarizing both the well-established and most recent directions within the field.  ...  We also provide a summary of common multi-task benchmarks.  ...  Each of these three blocks is made of a mix of convolutions, attention layers, and sparsely-gated mixture-of-experts layers.  ... 
arXiv:2009.09796v1 fatcat:d676uupucvgrbgnvsijqcexcqi

Message from the general chair

Benjamin C. Lee
2015 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)  
Learning-based Multi-Sieve Co-reference Resolution with Knowledge Lev Ratinov and Dan Roth Saturday 11:00am-11:30am -202 A (ICC) We explore the interplay of knowledge and structure in co-reference resolution  ...  Joint Learning for Coreference Resolution with Markov Logic Resolving "This-issue" Anaphora Varada Kolhatkar and Graeme Hirst Saturday 12:00pm-12:30pm -202 A (ICC) We annotate and resolve a particular  ...  label multi-lingual data with named entity tags.  ... 
doi:10.1109/ispass.2015.7095776 dblp:conf/ispass/Lee15 fatcat:ehbed6nl6barfgs6pzwcvwxria

No Language Left Behind: Scaling Human-Centered Machine Translation [article]

NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun (+27 others)
2022 arXiv   pre-print
More specifically, we developed a conditional compute model based on Sparsely Gated Mixture of Experts that is trained on data obtained with novel and effective data mining techniques tailored for low-resource  ...  Critically, we evaluated the performance of over 40,000 different translation directions using a human-translated benchmark, Flores-200, and combined human evaluation with a novel toxicity benchmark covering  ...  We thank Gloria Chang, Carole-Jean Wu and Ramya Raghavendra for helping us compute the CO 2 cost of our models. We thank Anjali Sridhar for help with FSDP.  ... 
arXiv:2207.04672v2 fatcat:gsbt3imt4bgodpmubpaq53onnm

Fine‐structure processing, frequency selectivity and speech perception in hearing‐impaired listeners

Olaf Strelcyk, Torsten Dau
2008 Journal of the Acoustical Society of America  
Natural, spontaneous speech often shows extreme reductions of many speech segments, to the point of apparent deletion.  ...  indicate that all three of these characteristics do affect listeners' percept of a consonant, but not sufficiently to completely account for the percept.  ...  The dispersive equation for the propagating wave lets introduce a criterion for the equilibrium state of the bubbly mixture with multi-size bubble population.  ... 
doi:10.1121/1.2935148 fatcat:nqyyia5pubamnhqgonegghrudm

Speaker comfort and increase of voice level in lecture rooms

Jonas Brunskog, Anders C. Gade, Gaspar Payà Bellester, Lilian Reig Calbo
2008 Journal of the Acoustical Society of America  
Natural, spontaneous speech often shows extreme reductions of many speech segments, to the point of apparent deletion.  ...  indicate that all three of these characteristics do affect listeners' percept of a consonant, but not sufficiently to completely account for the percept.  ...  The dispersive equation for the propagating wave lets introduce a criterion for the equilibrium state of the bubbly mixture with multi-size bubble population.  ... 
doi:10.1121/1.2934367 fatcat:xr6gp4ldo5bylnxytx2iumrdmi

Electroacoustical simulation of listening room acoustics for project ARCHIMEDES

Søren Bech
1989 Journal of the Acoustical Society of America  
These topics are reviewed with particular emphasis on the need for a comparable advance in translation of acoustic principles into building technologies. Sl Presented by: Ewart A.  ...  speech or music signals.  ...  The learning process uses synthesized noisy speech data, a mixture of pure speech data and noise data, to design reliable word reference vectors for the word spotting.  ... 
doi:10.1121/1.2027447 fatcat:7muohki2i5gktltdx2dhsiumze

The Multilingual Local in World Literature

Francesca Orsini
2015 Comparative Literature  
a structuring and generative principle and holds both local and cosmopolitan perspectives in view is more productive for world literature than approaches based only on cosmopolitan perspectives of circulation  ...  and recognition.  ...  Burcharth, Sophia Roosth, and Ruth Mack, as well as Sanjay Krishnan, Rebecca Gould, and Laetitia Zecchini, for probing questions and excellent feedback, and Peter Kornicki, as always.  ... 
doi:10.1215/00104124-3327481 fatcat:gwkqydkphrdzposibfummax4ay

Knowledge sharing for development [chapter]

2012 Development Centre Studies  
A related issue is the tendency for many to broadcast participant contributions to external audiences over the web, rather than leveraging the interactive potential of that technology to support knowledge  ...  The potential of new information and communication technologies (ICT) to facilitate a more inclusive model of iv One of the major findings of this research is the crucial role of incentives in shaping  ...  digital content on a multi-lingual web portal.  ... 
doi:10.1787/9789264173897-9-en fatcat:q5ifbfw7dfco3eyiywuqbkfdsm

On the Opportunities and Risks of Foundation Models [article]

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch (+102 others)
2022 arXiv   pre-print
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.  ...  Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties  ...  Foundation Models (CRFM), a center at Stanford University borne out of the Stanford Institute for Human-Centered Artificial Intelligence (HAI).  ... 
arXiv:2108.07258v3 fatcat:kohwrwk2ybf7fd7wsuz2gp65ki

Survey of Low-Resource Machine Translation [article]

Barry Haddow, Rachel Bawden, Antonio Valerio Miceli Barone, Jindřich Helcl, Alexandra Birch
2022 arXiv   pre-print
We present a summary of this topical research field and provide a description of the techniques evaluated by researchers in several recent shared tasks in low-resource MT.  ...  We present a survey covering the state of the art in low-resource machine translation research.  ...  Acknowledgements This work was partly funded by Rachel Bawden's chair in the PRAIRIE institute funded by the French national agency ANR as part of the "Investissements d'avenir" programme under the reference  ... 
arXiv:2109.00486v3 fatcat:5wof74vjy5gptcl5ornkd5j4ku

Survey of Low-Resource Machine Translation

Barry Haddow, Rachel Bawden, Antonio Valerio Miceli Barone, Jindřich Helcl, Alexandra Birch
2022 Computational Linguistics  
We present a survey covering the state of the art in low-resource machine translation research.  ...  There has been increasing interest in research addressing the challenge of producing useful translation models when very little translated training data is available.We present a summary of this topical  ...  Their two models, many to English and English to many, used a Sparsely Gated Mixture-of-Expert (MoE) models (Lepikhin et al. 2020) .  ... 
doi:10.1162/coli_a_00446 fatcat:mvpv6awfl5d3phudp5nd2cz2cy

Survey of Low-Resource Machine Translation

Haddow, Bawden, Miceli Barone, Helcl, Birch
2022 Zenodo  
We present a summary of this topical research field and provide a description of the techniques evaluated by researchers in several recent shared tasks in low-resource MT.  ...  We present a survey covering the state of the art in low-resource machine translation research.  ...  Their two models, many to English and English to many, used a Sparsely Gated Mixture-of-Expert (MoE) models (Lepikhin et al. 2020) .  ... 
doi:10.5281/zenodo.6672725 fatcat:ydiog4mdknglxjayk4rlpot5p4
« Previous Showing results 1 — 15 out of 106 results