Zhengan Chen
YOU?
Author Swipe
View article: From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics
From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics Open
Although transformer-based models have shown exceptional empirical performance, the fundamental principles governing their training dynamics are inadequately characterized beyond configuration-specific studies. Inspired by empirical eviden…
View article: Semi-Discrete in Time Method for Time-Dependent Equations by Random Neural Basis
Semi-Discrete in Time Method for Time-Dependent Equations by Random Neural Basis Open
Neural network-based solvers for partial differential equations (PDEs) have attracted considerable attention, yet they often face challenges in accuracy and computational efficiency. In this work, we focus on time-dependent PDEs and observ…
View article: On Multi-Stage Loss Dynamics in Neural Networks: Mechanisms of Plateau and Descent Stages
On Multi-Stage Loss Dynamics in Neural Networks: Mechanisms of Plateau and Descent Stages Open
The multi-stage phenomenon in the training loss curves of neural networks has been widely observed, reflecting the non-linearity and complexity inherent in the training process. In this work, we investigate the training dynamics of neural …
View article: On the dynamics of three-layer neural networks: initial condensation
On the dynamics of three-layer neural networks: initial condensation Open
Empirical and theoretical works show that the input weights of two-layer neural networks, when initialized with small values, converge towards isolated orientations. This phenomenon, referred to as condensation, indicates that the gradient…
View article: Phase Diagram of Initial Condensation for Two-layer Neural Networks
Phase Diagram of Initial Condensation for Two-layer Neural Networks Open
The phenomenon of distinct behaviors exhibited by neural networks under varying scales of initialization remains an enigma in deep learning research. In this paper, based on the earlier work by Luo et al.~\cite{luo2021phase}, we present a …
View article: Effect of warming on the carbon flux of the alpine wetland on the Qinghai–Tibet Plateau
Effect of warming on the carbon flux of the alpine wetland on the Qinghai–Tibet Plateau Open
Under the scenario of global warming, the response of greenhouse gas emissions from alpine wetlands remains unclear. In this study, fluxes of CO 2 and CH 4 were measured during daytime for the microtopographic features of hollows and hummo…