Filters








2 Hits in 1.2 sec

Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks [article]

Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, Douwe Kiela
2022 arXiv   pre-print
Dynatask is integrated with Dynabench, a research platform for rethinking benchmarking in AI that facilitates human and model in the loop data collection and evaluation.  ...  To create a task, users only need to write a short task configuration file from which the relevant web interfaces and model hosting infrastructure are automatically generated.  ...  Dynatask offers the functionalities of Dynabench to the broader research community by allowing them to easily create and host new AI tasks on the platform: it provides a one-stop shop for constructing  ... 
arXiv:2204.01906v1 fatcat:ako727taonhmhcf7mgkccqqxkq

Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks

Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, Douwe Kiela
2022 Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations   unpublished
Dynatask is integrated with Dynabench, a research platform for rethinking benchmarking in AI that facilitates human and model in the loop data collection and evaluation.  ...  To create a task, users only need to write a short task configuration file from which the relevant web interfaces and model hosting infrastructure are automatically generated.  ...  Dynatask offers the functionalities of Dynabench to the broader research community by allowing them to easily create and host new AI tasks on the platform: it provides a one-stop shop for constructing  ... 
doi:10.18653/v1/2022.acl-demo.17 fatcat:fofb2ncnfbed3d5kxb6bnmumbq