94 Hits in 4.3 sec

Show Your Work: Scratchpads for Intermediate Computation with Language Models [article]

Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, Augustus Odena
2021 arXiv   pre-print
On a series of increasingly complex tasks ranging from long addition to the execution of arbitrary programs, we show that scratchpads dramatically improve the ability of language models to perform multi-step  ...  In particular, we train transformers to perform multi-step computations by asking them to emit intermediate computation steps into a "scratchpad".  ...  ACKNOWLEDGMENTS We thank Fraser Greenlee for constructing the single-line programs dataset, and Kevin Murphy for bringing this dataset to our attention.  ... 
arXiv:2112.00114v1 fatcat:qk5u752x5bdvneqemsuoy7eqke


Jim Gray, Surajit Chaudhuri, Adam Bosworth, Andrew Layman, Don Reichart, Murali Venkatrao, Frank Pellow, Hamid Pirahesh
2012 Data mining and knowledge discovery  
This paper (1) explains the cube and roll-up operators, (2) shows how they fit in SQL, (3) explains how users can define new aggregate functions for cubes, and (4) discusses efficient techniques to compute  ...  Super-aggregates are computed by aggregating the N -cube to lower dimensional spaces.  ...  Finally, the end() call computes the aggregate from the scratchpad values, deallocates the scratchpad and returns the result.  ... 
doi:10.1023/a:1009726021843 fatcat:ylzvngqvfrglrkmg3e7me6ateq

A domain-specific supercomputer for training deep neural networks

Norman P. Jouppi, Doe Hyun Yoon, George Kurian, Sheng Li, Nishant Patil, James Laudon, Cliff Young, David Patterson
2020 Communications of the ACM  
Google's TPU supercomputers train deep neural networks 50x faster than general-purpose supercomputers running a high-performance computing benchmark.  ...  Table 4 shows the speedup from the fusion optimization on 2D data is from 1.8 to 6.3. The TF intermediate form for XLA has thousands of ops.  ...  Like many systems that map from domain-specific languages to code, XLA integrates a high-level library and a compiler. A TF front end generates code in an intermediate representation for XLA.  ... 
doi:10.1145/3360307 fatcat:xomnv3wdebdxphccmfhccapcwa

A review of mathematica

Richard J. Fateman
1992 Journal of symbolic computation  
The Mathematica computer system is reviewed from the perspective of its contributions to symbolic and algebraic computation, as well as its stated goals.  ...  Using PostScript as a device-independent intermediate form for plots is somewhat novel; other programs (including Maple and Macsyma) tend to use PostScript only for communicating with hard-copy devices  ...  Group 1990 , and AXIOM , (previously referred to as Scratchpad II (Computer Algebra Group (1988) )).  ... 
doi:10.1016/s0747-7171(10)80011-2 fatcat:m54bnhziabga7bog4uwaykzzbe

Amanuensis: The Programmer's Apprentice [article]

Thomas Dean, Maurice Chiang, Marcus Gomez, Nate Gruver, Yousef Hindy, Michelle Lam, Peter Lu, Sophia Sanchez, Rohun Saxena, Michael Smith, Lucy Wang, Catherine Wong
2018 arXiv   pre-print
As a concrete example we focus on digital assistants that learn from continuous dialog with an expert software engineer while providing initial value as powerful analytical, computational and mathematical  ...  Over time these savants learn cognitive strategies (domain-relevant problem solving skills) and develop intuitions (heuristics and the experience necessary for applying them) by learning from their expert  ...  works of its own invention, novel but consistent with your taste and sensibilities.  ... 
arXiv:1807.00082v2 fatcat:piwexqa2xvgg5ec5xwkswstswy

Prutor: A System for Tutoring CS1 and Collecting Student Programs for Analysis [article]

Rajdeep Das, Umair Z. Ahmed, Amey Karkare, Sumit Gulwani
2016 arXiv   pre-print
This system thus serves as a platform for tutoring as well as data collection for researchers.  ...  These intermediate versions of the student programs provide the instructors (and data analysts) a view of the students' approach to solving programming problems.  ...  In general, in order to allow the system to work for a new imperative programming language, the only modification required is to change the configuration files.  ... 
arXiv:1608.03828v1 fatcat:r3wstog2d5ehba6jscmlqi5pqq

PIMP My Many-Core: Pipeline-Integrated Message Passing

Jörg Mische, Martin Frieb, Alexander Stegmeier, Theo Ungerer
2020 International journal of parallel programming  
To improve the scalability, several many-core architectures use message passing instead of shared memory accesses for communication.  ...  PIMP was integrated into the FPGA model of a classical single issue RISC pipeline to show the small hardware costs compared with other register-level message passing architectures.  ...  For example, one of the first message passing many-cores, the Intel Single-chip Cloud Computer (SCC) [13] , provides so-called Message Passing Buffers (MPBs), small scratchpad memories tightly coupled  ... 
doi:10.1007/s10766-020-00685-9 fatcat:hyabinpgkjdmjdyu2347oh5ukm

NVMExplorer: A Framework for Cross-Stack Comparisons of Embedded Non-Volatile Memories [article]

Lillian Pentecost, Alexander Hankin, Marco Donato, Mark Hempstead, Gu-Yeon Wei, David Brooks
2022 arXiv   pre-print
This work evaluates eNVM-based storage for a range of application and system contexts including machine learning on the edge, graph analytics, and general purpose cache hierarchy, in addition to describing  ...  Repeated off-chip memory accesses to DRAM drive up operating power for data-intensive applications, and SRAM technology scaling and leakage power limits the efficiency of embedded memories.  ...  Traffic patterns are extracted from throughput and accesses reported for the compute stream of a domain-specific graph processing accelerator utilizing an 8MB eDRAM scratchpad [52] .  ... 
arXiv:2109.01188v2 fatcat:h7jx7rulzbgtjgb55zzp3cdi2a

Concurrent Computing in the Many-core Era (Dagstuhl Seminar 15021)

Michael Philippsen, Pascal Felber, Michael L. Scott, J. Eliot B. Moss, Marc Herbstritt
2015 Dagstuhl Reports  
This seminar is a successor to Dagstuhl Seminars 08241 "Transactional memory: From implementation to application" and 12161 "Abstractions for scalable multicore computing", respectively held in June 2008  ...  This report documents the program and the outcomes of Dagstuhl Seminar 15021 "Concurrent computing in the many-core era".  ...  The work leads to several research questions about parallel programming models and runtimes for popular dynamic languages with none or very limited support for parallel execution such as JavaScript, Ruby  ... 
doi:10.4230/dagrep.5.1.1 dblp:journals/dagstuhl-reports/PhilippsenFSM15 fatcat:owcmta65hzb5vmglwq3dwzbehy

BioVeL: a virtual laboratory for data analysis and modelling in biodiversity science and ecology

Alex R. Hardisty, Finn Bacall, Niall Beard, Maria-Paula Balcázar-Vargas, Bachir Balech, Zoltán Barcza, Sarah J. Bourlat, Renato De Giovanni, Yde de Jong, Francesca De Leo, Laura Dobor, Giacinto Donvito (+40 others)
2016 BMC Ecology  
We developed and improved more than 60 Web services with significant potential in many different kinds of data analysis and modelling tasks.  ...  ; for on-line collaboration through sharing of workflows and workflow runs; for experiment documentation through reproducibility and repeatability; and for computational support via seamless connections  ...  We thank you for your contribution. We acknowledge the contribution of Yannick Legré, who contributed to the conception and design of the work but ultimately was unable to participate in the project.  ... 
doi:10.1186/s12898-016-0103-y pmid:27765035 pmcid:PMC5073428 fatcat:kphzgb67bzh4tdnwchbkvf2nvq

Can language models learn from explanations in context? [article]

Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James L. McClelland, Jane X. Wang, Felix Hill
2022 arXiv   pre-print
Large language models can perform new tasks by adapting to a few in-context examples. For humans, rapid learning from examples can benefit from explanations that connect examples to task principles.  ...  We then show that explanations tuned for performance on a small validation set offer substantially larger benefits; building a prompt by selecting examples and explanations together substantially improves  ...  Acknowledgements We thank Dani Yogatama and Neil Rabinowitz for helpful comments and suggestions, as well as the team that trained the language models.  ... 
arXiv:2204.02329v1 fatcat:iwqbt3lpmnahtcspl2vmig5zom

A framework for constructing cognition ontologies using WordNet, FrameNet, and SUMO

Stephen K. Reed, Adam Pease
2015 Cognitive Systems Research  
The next section discusses the objectives of science ontologies and includes examples for psychoses and for 17 emotion.  ...  This article 12 supports that objective by proposing a framework for constructing cognition ontologies by using WordNet, FrameNet, and the Sug-13 gested Upper Merged Ontology (SUMO).  ...  From Barrouillet et al. (2011).The rationale for this claim is that interference in Kahn-1095 eman's capacity model occurs when the demand for capac-1096 ity exceeds the supply.  ... 
doi:10.1016/j.cogsys.2014.06.001 fatcat:dpiwlklhgfcxnjn3ru32b646tm

Identifying middle school students' challenges in computational thinking-based science learning

Satabdi Basu, Gautam Biswas, Pratim Sengupta, Amanda Dickes, John S. Kinnebrew, Douglas Clark
2016 Research and Practice in Technology Enhanced Learning  
In this paper, we present a research study with CTSiM (Computational Thinking in Simulation and Modeling)-computational thinking-based learning environment for K-12 science, where students build and simulate  ...  We investigate a set of core challenges (both computational and science domain related) that middle school students face when working with CTSiM, how these challenges evolve across different modeling activities  ...  This work was supported by the NSF (NSF Cyber-learning grant #1124175 and #1441542).  ... 
doi:10.1186/s41039-016-0036-2 pmid:30613246 pmcid:PMC6302855 fatcat:7eksr6d635h63nzmoxfhqbsxo4

A memory theoretic approach for investigating the roles of language and intuition in mathematical thinking activities [article]

Manouchehr Zaker
2019 arXiv   pre-print
Ongoing mathematical activities both elementary and advanced, demand working memory resources.  ...  But these beliefs are usually in a big contrast with the recent cognitive theoretic findings concerning mathematics.  ...  Hence, lengthy and complicated computations with infinite sums and series are replaced by formal computations with integrals.  ... 
arXiv:1910.00003v1 fatcat:m3nfald5yvd25ioexblh3vswjq

Expressing Parallelism [chapter]

James Reinders, Ben Ashbaugh, James Brodman, Michael Kinsner, John Pennycook, Xinmin Tian
2020 Data Parallel C++  
We already know how to place code (Chapter 10.1007/978-1-4842-5574-2_2) and data (Chapter 10.1007/978-1-4842-5574-2_3) on a device—all we must do now is engage in the art of deciding what to do with it  ...  explicitly managed "scratchpad" memories.  ...  Choosing a Kernel Form Choosing between the different kernel forms is largely a matter of personal preference and heavily influenced by prior experience with other parallel programming models and languages  ... 
doi:10.1007/978-1-4842-5574-2_4 fatcat:ppar5yv5nfgbjiclwwmw74eeua
« Previous Showing results 1 — 15 out of 94 results