arXiv (Cornell University)
Hyper-Compression: Model Compression via Hyperfunction
September 2024 • Fenglei Fan, Juntong Fan, Dayang Wang, Jingbo Zhang, Zelin Dong, Zhang Shi-jun, Ge Wang, Tieyong Zeng
The rapid growth of large models' size has far outpaced that of computing resources. To bridge this gap, encouraged by the parsimonious relationship between genotype and phenotype in the brain's growth and development, we propose the so-called hyper-compression that turns the model compression into the issue of parameter representation via a hyperfunction. Specifically, it is known that the trajectory of some low-dimensional dynamic systems can fill the high-dimensional space eventually. Thus, hyper-compression, u…