Jiyan Yang
YOU?
Author Swipe
View article: Enhancing Embedding Representation Stability in Recommendation Systems with Semantic ID
Enhancing Embedding Representation Stability in Recommendation Systems with Semantic ID Open
The exponential growth of online content has posed significant challenges to ID-based models in industrial recommendation systems, ranging from extremely high cardinality and dynamically growing ID space, to highly skewed engagement distri…
View article: External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation
External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation Open
Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. H…
View article: The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit
The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit Open
The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization fr…
View article: A Collaborative Ensemble Framework for CTR Prediction
A Collaborative Ensemble Framework for CTR Prediction Open
Recent advances in foundation models have established scaling laws that enable the development of larger models to achieve enhanced performance, motivating extensive research into large-scale recommendation models. However, simply increasi…
View article: InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Prediction
InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Prediction Open
Click-through rate (CTR) prediction, which predicts the probability of a user clicking an ad, is a fundamental task in recommender systems. The emergence of heterogeneous information, such as user profile and behavior sequences, depicts us…
View article: Hierarchical Structured Neural Network: Efficient Retrieval Scaling for Large Scale Recommendation
Hierarchical Structured Neural Network: Efficient Retrieval Scaling for Large Scale Recommendation Open
Retrieval, the initial stage of a recommendation system, is tasked with down-selecting items from a pool of tens of millions of candidates to a few thousands. Embedding Based Retrieval (EBR) has been a typical choice for this problem, addr…
View article: Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale
Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale Open
Neural Architecture Search (NAS) has demonstrated its efficacy in computer vision and potential for ranking systems. However, prior work focused on academic problems, which are evaluated at small scale under well-controlled fixed baselines…
View article: AutoML for Large Capacity Modeling of Meta's Ranking Systems
AutoML for Large Capacity Modeling of Meta's Ranking Systems Open
Web-scale ranking systems at Meta serving billions of users is complex. Improving ranking models is essential but engineering heavy. Automated Machine Learning (AutoML) can potentially release engineers from labor intensive work of tuning …
View article: AutoML for Large Capacity Modeling of Meta's Ranking Systems
AutoML for Large Capacity Modeling of Meta's Ranking Systems Open
Web-scale ranking systems at Meta serving billions of users is complex. Improving ranking models is essential but engineering heavy. Automated Machine Learning (AutoML) can release engineers from labor intensive work of tuning ranking mode…
View article: Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale
Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale Open
Neural Architecture Search (NAS) has demonstrated its efficacy in computer vision and potential for ranking systems. However, prior work focused on academic problems, which are evaluated at small scale under well-controlled fixed baselines…
View article: Towards the Better Ranking Consistency: A Multi-task Learning Framework for Early Stage Ads Ranking
Towards the Better Ranking Consistency: A Multi-task Learning Framework for Early Stage Ads Ranking Open
Dividing ads ranking system into retrieval, early, and final stages is a common practice in large scale ads recommendation to balance the efficiency and accuracy. The early stage ranking often uses efficient models to generate candidates o…
View article: Corrigendum: Predicting the recurrence and overall survival of patients with glioma based on histopathological images using deep learning
Corrigendum: Predicting the recurrence and overall survival of patients with glioma based on histopathological images using deep learning Open
[This corrects the article DOI: 10.3389/fneur.2023.1100933.].
View article: Correlation of ABO blood groups with treatment response and efficacy in infants with persistent pulmonary hypertension of the newborn treated with inhaled nitric oxide
Correlation of ABO blood groups with treatment response and efficacy in infants with persistent pulmonary hypertension of the newborn treated with inhaled nitric oxide Open
Objective Not all infants with persistent pulmonary hypertension of the newborn (PPHN) respond to inhaled nitric oxide (iNO) therapy, as it is known to improve oxygenation in only 50% to 60% of cases. In this study, we investigated whether…
View article: AdaTT: Adaptive Task-to-Task Fusion Network for Multitask Learning in Recommendations
AdaTT: Adaptive Task-to-Task Fusion Network for Multitask Learning in Recommendations Open
Multi-task learning (MTL) aims to enhance the performance and efficiency of machine learning models by simultaneously training them on multiple tasks. However, MTL research faces two challenges: 1) effectively modeling the relationships be…
View article: Predicting the recurrence and overall survival of patients with glioma based on histopathological images using deep learning
Predicting the recurrence and overall survival of patients with glioma based on histopathological images using deep learning Open
Background A deep learning (DL) model based on representative biopsy tissues can predict the recurrence and overall survival of patients with glioma, leading to optimized personalized medicine. This research aimed to develop a DL model bas…
View article: DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction
DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction Open
Learning feature interactions is important to the model performance of online advertising services. As a result, extensive efforts have been devoted to designing effective architectures to learn feature interactions. However, we observe th…
View article: High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models.
High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models. Open
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-…
View article: Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models Open
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-…
View article: CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery Open
The paper proposes and optimizes a partial recovery training system, CPR, for recommendation models. CPR relaxes the consistency requirement by enabling non-failed nodes to proceed without loading checkpoints when a node fails during train…
View article: Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data Open
Large scale deep learning provides a tremendous opportunity to improve the quality of content recommendation systems by employing both wider and deeper models, but this comes at great infrastructural cost and carbon footprint in modern dat…
View article: Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems Open
Modern deep learning-based recommendation systems exploit hundreds to\nthousands of different categorical features, each with millions of different\ncategories ranging from clicks to posts. To respect the natural diversity\nwithin the cate…
View article: Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems Open
Large-scale training is important to ensure high performance and accuracy of machine-learning models. At Facebook we use many different models, including computer vision, video and language models. However, in this paper we focus on the de…
View article: ShadowSync: Performing Synchronization in the Background for Highly Scalable Distributed Training
ShadowSync: Performing Synchronization in the Background for Highly Scalable Distributed Training Open
Recommendation systems are often trained with a tremendous amount of data, and distributed training is the workhorse to shorten the training time. While the training throughput can be increased by simply adding more workers, it is also inc…
View article: Post-Training 4-bit Quantization on Embedding Tables
Post-Training 4-bit Quantization on Embedding Tables Open
Continuous representations have been widely adopted in recommender systems where a large number of entities are represented using embedding vectors. As the cardinality of the entities increases, the embedding components can easily contain …
View article: Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems Open
Embedding representations power machine intelligence in many applications, including recommendation systems, but they are space intensive -- potentially occupying hundreds of gigabytes in large-scale settings. To help manage this outsized …
View article: A Study of BFLOAT16 for Deep Learning Training
A Study of BFLOAT16 for Deep Learning Training Open
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training across image classification, speech recognition, language modeli…