Pascal Tikeng Notsawo
YOU?
Author Swipe
View article: Lost in Translation: The Algorithmic Gap Between LMs and the Brain
Lost in Translation: The Algorithmic Gap Between LMs and the Brain Open
Language Models (LMs) have achieved impressive performance on various linguistic tasks, but their relationship to human language processing in the brain remains unclear. This paper examines the gaps and overlaps between LMs and the brain a…
View article: Stochastic Average Gradient : A Simple Empirical Investigation
Stochastic Average Gradient : A Simple Empirical Investigation Open
Despite the recent growth of theoretical studies and empirical successes of neural networks, gradient backpropagation is still the most widely used algorithm for training such networks. On the one hand, we have deterministic or full gradie…
View article: Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok
Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok Open
This paper focuses on predicting the occurrence of grokking in neural networks, a phenomenon in which perfect generalization emerges long after signs of overfitting or memorization are observed. It has been reported that grokking can only …