Explanipedia

Let Humanoids Hike! Integrative Skill Development on Complex Trails Open

Kwan-Yee Lin, Stella X. Yu · 2025

Hiking on complex trails demands balance, agility, and adaptive decision-making over unpredictable terrain. Current humanoid research remains fragmented and inadequate for hiking: locomotion focuses on motor skills without long-term goals …

TimeWalker: Personalized Neural Space for Lifelong Head Avatars Open

Dongwei Pan, Yang Li, Hongsheng Li, Kwan-Yee Lin · 2024

We present TimeWalker, a novel framework that models realistic, full-scale 3D head avatars of a person on lifelong scale. Unlike current human head avatar pipelines that capture identity at the momentary level(e.g., instant photography or …

Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior Open

Fan Lü, Kwan-Yee Lin, Yan Xu, Hongsheng Li, Guang Chen , et al. · 2024

Text-to-3D generation has achieved remarkable success via large-scale text-to-image diffusion models. Nevertheless, there is no paradigm for scaling up the methodology to urban scale. Urban scenes, characterized by numerous elements, intri…

CosmicMan: A Text-to-Image Foundation Model for Humans Open

Shikai Li, Jianglin Fu, Kaiyuan Liu, Wentao Wang, Kwan-Yee Lin , et al. · 2024

We present CosmicMan, a text-to-image foundation model specialized for generating high-fidelity human images. Unlike current general-purpose foundation models that are stuck in the dilemma of inferior quality and text-image misalignment fo…

RNNPose: 6-DoF Object Pose Estimation via Recurrent Correspondence Field Estimation and Pose Optimization Open

Yan Xu, Kwan-Yee Lin, Guofeng Zhang, Xiaogang Wang, Hongsheng Li · 2024

6-DoF object pose estimation from a monocular image is a challenging problem, where a post-refinement procedure is generally needed for high-precision estimation. In this paper, we propose a framework, dubbed RNNPose, based on a recurrent …

Parameterization-driven Neural Surface Reconstruction for Object-oriented Editing in Neural Rendering Open

Baixin Xu, Jiangbei Hu, Fei Hou, Kwan-Yee Lin, Wayne Wu , et al. · 2023

The advancements in neural rendering have increased the need for techniques that enable intuitive editing of 3D objects represented as neural implicit surfaces. This paper introduces a novel neural algorithm for parameterizing neural impli…

UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation Open

Jianglin Fu, Shikai Li, Yuming Jiang, Kwan-Yee Lin, Wayne Wu , et al. · 2023

Human generation has achieved significant progress. Nonetheless, existing methods still struggle to synthesize specific regions such as faces and hands. We argue that the main reason is rooted in the training data. A holistic human dataset…

Urban Radiance Field Representation with Deformable Neural Mesh Primitives Open

Fan Lü, Yan Xu, Guang Chen, Hongsheng Li, Kwan-Yee Lin , et al. · 2023

Neural Radiance Fields (NeRFs) have achieved great success in the past few years. However, most current methods still require intensive resources due to ray marching-based rendering. To construct urban-level radiance fields efficiently, we…

DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering Open

Cheng Wei, Ruixiang Chen, Wanqi Yin, Siming Fan, Keyu Chen , et al. · 2023

Realistic human-centric rendering plays a key role in both computer vision and computer graphics. Rapid progress has been made in the algorithm aspect over the years, yet existing human-centric rendering datasets and benchmarks are rather …

RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars Open

Dongwei Pan, Zhuo Long, Jingtan Piao, Huiwen Luo, Wei Cheng , et al. · 2023

Synthesizing high-fidelity head avatars is a central problem for computer vision and graphics. While head avatar synthesis algorithms have advanced rapidly, the best ones still face great obstacles in real-world scenarios. One of the vital…

MonoHuman: Animatable Human Neural Field from Monocular Video Open

Zhengming Yu, Wei Cheng, Xian Liu, Wayne Wu, Kwan-Yee Lin · 2023

Animating virtual avatars with free-view control is crucial for various applications like virtual reality and digital entertainment. Previous studies have attempted to utilize the representation power of the neural radiance field (NeRF) to…

Deformable Model-Driven Neural Rendering for High-Fidelity 3D Reconstruction of Human Heads Under Low-View Settings Open

Baixin Xu, Jiarui Zhang, Kwan-Yee Lin, Qian Chen, Ying He · 2023

Reconstructing 3D human heads in low-view settings presents technical challenges, mainly due to the pronounced risk of overfitting with limited views and high-frequency signals. To address this, we propose geometry decomposition and adopt …

StyleGAN-Human: A Data-Centric Odyssey of Human Generation Open

Jianglin Fu, Shikai Li, Yuming Jiang, Kwan-Yee Lin, Qian Chen , et al. · 2022

Unconditional human image generation is an important task in vision and graphics, which enables various applications in the creative industry. Existing studies in this field mainly focus on "network engineering" such as designing new compo…

Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis Open

Wei Cheng, Xu Su, Jingtan Piao, Qian Chen, Wayne Wu , et al. · 2022

This work targets at using a general deep learning framework to synthesize free-viewpoint images of arbitrary human performers, only requiring a sparse number of camera views as inputs and skirting per-case fine-tuning. The large variation…

Simulating Fluids in Real-World Still Images Open

Siming Fan, Jingtan Piao, Qian Chen, Kwan-Yee Lin, Hongsheng Li · 2022

In this work, we tackle the problem of real-world fluid animation from a still image. The key of our system is a surface-based layered representation deriving from video decomposition, where the scene is decoupled into a surface fluid laye…

Learning a Structured Latent Space for Unsupervised Point Cloud Completion Open

Yingjie Cai, Kwan-Yee Lin, Chao Zhang, Qiang Wang, Xiaogang Wang , et al. · 2022

Unsupervised point cloud completion aims at estimating the corresponding complete point cloud of a partial point cloud in an unpaired manner. It is a crucial but challenging problem since there is no paired partial-complete supervision tha…

RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization Open

Yan Xu, Kwan-Yee Lin, Guofeng Zhang, Xiaogang Wang, Hongsheng Li · 2022

6-DoF object pose estimation from a monocular image is challenging, and a post-refinement procedure is generally needed for high-precision estimation. In this paper, we propose a framework based on a recurrent neural network (RNN) for obje…

Inverting Generative Adversarial Renderer for Face Reconstruction Open

Jingtan Piao, Keqiang Sun, Kwan-Yee Lin, Quan Wang, Hongsheng Li · 2021

Given a monocular face image as input, 3D face geometry reconstruction aims to recover a corresponding 3D face mesh. Recently, both optimization-based and learning-based face reconstruction methods have taken advantage of the emerging diff…

Semantic Scene Completion via Integrating Instances and Scene in-the-Loop Open

Yingjie Cai, Xuesong Chen, Chao Zhang, Kwan-Yee Lin, Xiaogang Wang , et al. · 2021

Semantic Scene Completion aims at reconstructing a complete 3D scene with precise voxel-wise semantics from a single-view depth or RGBD image. It is a crucial but challenging problem for indoor scene understanding. In this work, we present…

SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks Open

Yan Xu, Zhaoyang Huang, Kwan-Yee Lin, Xinge Zhu, Jianping Shi , et al. · 2020

Recent learning-based LiDAR odometry methods have demonstrated their competitiveness. However, most methods still face two substantial challenges: 1) the 2D projection representation of LiDAR data cannot effectively encode 3D structures fr…

Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation Open

Xiaokang Chen, Kwan-Yee Lin, Jingbo Wang, Wayne Wu, Chen Qian , et al. · 2020

Depth information has proven to be a useful cue in the semantic segmentation of RGB-D images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-a…

3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior Open

Xiaokang Chen, Kwan-Yee Lin, Qian Chen, Gang Zeng, Hongsheng Li · 2020

The goal of the Semantic Scene Completion (SSC) task is to simultaneously predict a completed 3D voxel representation of volumetric occupancy and semantic labels of objects in the scene from a single-view observation. Since the computation…

TRB: A Novel Triplet Representation for Understanding 2D Human Body Open

Haodong Duan, Kwan-Yee Lin, Sheng Jin, Wentao Liu, Chen Qian , et al. · 2019

Human pose and shape are two important components of 2D human body. However, how to efficiently represent both of them in images is still an open question. In this paper, we propose the Triplet Representation for Body (TRB) -- a compact 2D…

Make a Face: Towards Arbitrary High Fidelity Face Manipulation Open

Shengju Qian, Kwan-Yee Lin, Wayne Wu, Yangxiaokang Liu, Quan Wang , et al. · 2019

Recent studies have shown remarkable success in face manipulation task with the advance of GANs and VAEs paradigms, but the outputs are sometimes limited to low-resolution and lack of diversity. In this work, we propose Additive Focal Vari…

Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation Open

Xipeng Chen, Kwan-Yee Lin, Wentao Liu, Chen Qian, Xiaogang Wang , et al. · 2019

Recent studies have shown remarkable advances in 3D human pose estimation from monocular images, with the help of large-scale in-door 3D datasets and sophisticated network architectures. However, the generalizability to different environme…

Hallucinated-IQA: No-Reference Image Quality Assessment via Adversarial Learning Open

Kwan-Yee Lin, Guanxiang Wang · 2018

No-reference image quality assessment (NR-IQA) is a fundamental yet challenging task in low-level computer vision community. The difficulty is particularly pronounced for the limited information, for which the corresponding reference for c…

Kwan-Yee Lin YOU? Author Swipe