LLaMA: Open and Efficient Foundation Language Models Article Swipe

PDF

Related Concepts

Foundation (evidence) Computer science Ranging Language model State (computer science) Artificial intelligence Programming language Archaeology Geography Telecommunications

Hugo Touvron , Thibaut Lavril , Gautier Izacard , Xavier Martinet , Marie-Anne Lachaux , Timothée Lacroix , Baptiste Rozière , Naman Goyal , Eric Hambro , Faisal Azhar , Aurelien Rodriguez , Armand Joulin , Édouard Grave , Guillaume Lample ·

YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2302.13971 · OA: W4322718191

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.