main webpage
W Topic
Machine Learning
Findings of the Association for Computational Linguistics: NAACL 2022
LongT5: Efficient Text-To-Text Transformer for Long Sequences
2022
Recent work has shown that either (1) increasing the input length or (2) increasing model size can improve the performance of Transformer-based neural models. In this paper, we present LongT5, a new model that explores the effects of scaling both the input le…
Article

Machine Learning

Study of algorithms that improve automatically through experience

Machine learning (ML) is a field of study in artificial intelligence concerned with the development and study of statistical algorithms that can learn from data and generalise to unseen data, and thus perform tasks without explicit instructions. Within a subdiscipline in machine learning, advances in the field of deep learning have allowed neural networks, a class of statistical algorithms, to surpass many previous machine learning approaches in performance.

ML finds application in many fields, including natural language processing, computer vision, speech recognition, email filtering, agriculture, and medicine. The application of ML to business problems is known as predictive analytics.

Exploring foci of:
Findings of the Association for Computational Linguistics: NAACL 2022
LongT5: Efficient Text-To-Text Transformer for Long Sequences
2022
Recent work has shown that either (1) increasing the input length or (2) increasing model size can improve the performance of Transformer-based neural models. In this paper, we present LongT5, a new model that explores the effects of scaling both the input length and model size at the same time. Specifically, we integrate attention ideas from long-input transformers (ETC), and adopt pre-training strategies from summarization pre-training (PEGASUS) into the scalable T5 architecture. The result is a new attention me…
Click Machine Learning Vs:
Computer Science
Transformer
Architecture
Artificial Intelligence
Database
Engineering
Art
Visual Arts
Electrical Engineering
Click Machine Learning Vs:
Voltage