Ye Yuan

I'm a Senior Research Scientist in the Learning and Perception Team at NVIDIA Research. I received my Ph.D. in Robotics from Carnegie Mellon University in 2022, where I was advised by Prof. Kris Kitani. I also earned my M.S. in computer science at CMU in 2016, where I worked with Prof. Stelian Coros. I obtained my B.E. in computer science and technology from Zhejiang University in 2015. My research has been supported by the Qualcomm Innovation Fellowship and the NVIDIA Graduate Fellowship.

Email | CV | Google Scholar | Twitter | Github

News

July	2023	Two papers accepted to CVPR 2024.
July	2023	Two papers accepted to ICCV 2023. See you in Paris!
Mar	2023	One paper on learning digital tennis player from videos accepted to SIGGRAPH 2023 (Best Paper Honorable Mention).
Feb	2023	One paper on simulating pedestrian motions accepted to CVPR 2023.
Sep	2022	One paper on embodied human pose estimation accepted to NeurIPS 2022.
May	2022	Joined NVIDIA Research as a Research Scientist.
Apr	2022	Defended my Ph.D. thesis Unified Simulation, Perception, and Generation of Human Behavior.
Mar	2022	One paper on global human mesh recovery accepted to CVPR 2022 with an Oral Presentation.
Jan	2022	One paper on efficient automatic agent design accepted to ICLR 2022 with an Oral Presentation.
Jan	2022	Invited Talk at MPI Perceiving Systems.
Sep	2021	One paper on kinematics-guided control accepted to NeurIPS 2021.
July	2021	One paper on multi-agent forecasting accepted to ICCV 2021.
May	2021	Starting my internship at NVIDIA AI.
Apr	2021	Invited Talk at ETH Zurich, Computer Vision and Learning Group.
Apr	2021	Invited Talk at "Machine Learning and Optimal Control" class, University of Alabama.
Mar	2021	Received an outstanding reviewer award by ICLR 2021.
Feb	2021	One paper on physically-plausible human pose estimation accepted to CVRP 2021 with an Oral Presentation.
Feb	2021	One paper accepted to ICRA 2021.
Feb	2021	Invited Talk at 16th CSL student conference.
Feb	2021	Invited Talk at UIUC Robotics Seminar.
Dec	2020	Invited Talks at Qualcomm and Wayve.
Dec	2020	Honored to receive 2021 NVIDIA Graduate Fellowship.
Sep	2020	One paper on Residual Force Control accepted to NeurIPS 2020.
Aug	2020	Honored to receive 2020 Qualcomm Innovation Fellowship.
July	2020	Two papers accepted to ECCV 2020.
May	2020	Starting my internship at Facebook Reality Lab Pittsburgh.
Feb	2020	Two papers accepted to CVPR 2020, one with an Oral Presentation.
Dec	2019	Paper Diverse Trajectory Forecasting with Determinantal Point Processes accepted to ICLR 2020.
July	2019	Paper Ego-Pose Estimation and Forecasting as Real-Time PD Control accepted to ICCV 2019.
July	2018	Paper 3D Ego-Pose Estimation via Imitation Learning accepted to ECCV 2018.

Research

My research lies at the intersection of computer vision, machine learning, and robotics. My current focus is on digital humans, 3D generative AI, and embodied AI. Example topics: 3D human avatar generation, human motion generation, humanoid control, etc.

	AGG: Amortized Generative 3D Gaussians for Single Image to 3D Dejia Xu, Ye Yuan, Morteza Mardani, Sifei Liu, Jiaming Song, Zhangyang Wang, Arash Vahdat arXiv, 2024 project page \| arXiv \| video
	GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning Ye Yuan, Xueting Li, Yangyi Huang, Shalini De Mello, Koki Nagano, Jan Kautz, Umar Iqbal (Equal Contribution) CVPR, 2024 (Highlight)* project page \| arXiv \| video
	PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios Jingbo Wang, Zhengyi Luo, Ye Yuan, Yixuan Li, Bo Dai CVPR, 2024 project page \| arXiv \| video \| code
	PACE: Human and Camera Motion Estimation from in-the-wild Videos Muhammed Kocabas, Ye Yuan, Pavlo Molchanov, Yunrong Guo, Michael Black, Otmar Hilliges, Jan Kautz, Umar Iqbal 3DV, 2024 (Spotlight Presentation) project page \| arXiv \| video
	PhysDiff: Physics-Guided Human Motion Diffusion Model Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz ICCV, 2023 (Oral Presentation) project page \| arXiv \| video
	Learning Human Dynamics in Autonomous Driving Scenarios Jingbo Wang, Ye Yuan, Zhengyi Luo, Kevin Xie, Dahua Lin, Umar Iqbal, Sanja Fidler, Sameh Khamis ICCV, 2023 arXiv
	Learning Physically Simulated Tennis Players from Broadcast Videos Haotian Zhang, Ye Yuan, Viktor Makoviychuk, Yunrong Guo, Sanja Fidler, Xue Bin Peng, Kayvon Fatahalian SIGGRAPH, 2023 (Best Paper Honorable Mention) project page \| paper \| video
	Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Sanja Fidler, Or Litany CVPR, 2023 project page \| arXiv \| demo
	RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay, Bowen Wen, Ye Yuan, Stephen Tyree, Charles Loop, Alexander Schwing, Stan Birchfield ICRA, 2023 project page \| arXiv \| video \| data
	Embodied Scene-aware Human Pose Estimation Zhengyi Luo, Shun Iwase, Ye Yuan, Kris Kitani NeurIPS, 2022 project page \| arXiv \| video
	Unified Simulation, Perception, and Generation of Human Behavior Ye Yuan Ph.D. Thesis, Robotics Institute, CMU, 2022 arXiv
	GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras Ye Yuan, Umar Iqbal, Pavlo Molchanov, Kris Kitani, Jan Kautz CVPR, 2022 (Oral Presentation - Top 4.2%) project page \| arXiv \| video \| code
	Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris Kitani ICLR, 2022 (Oral Presentation - Top 1.6%) project page \| arXiv \| openreview \| code
	Online No-regret Model-Based Meta RL for Personalized Navigation Yuda Song, Ye Yuan, Wen Sun, Kris Kitani Learning for Dynamics & Control (L4DC), 2022 paper
	Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation Zhengyi Luo, Ryo Hachiuma, Ye Yuan, Kris Kitani NeurIPS, 2021 project page \| arXiv \| video \| code
	AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting Ye Yuan, Xinshuo Weng, Yanglan Ou, Kris Kitani ICCV, 2021 project page \| arXiv \| code
	SimPoE: Simulated Character Control for 3D Human Pose Estimation Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, Jason Saragih CVPR, 2021 (Oral Presentation - Top 4.2%) project page \| arXiv \| talk \| video
	PTP: Parallelized 3D Tracking and Prediction with Graph Neural Networks and Diversity Sampling Xinshuo Weng, Ye Yuan*, Kris Kitani (Equal Contribution) RA-L and ICRA, 2021 (Best Student Paper Candidate < 2%) project page \| arXiv \| code
	Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis Ye Yuan, Kris Kitani NeurIPS, 2020 project page \| arXiv \| video \| code
	DLow: Diversifying Latent Flows for Diverse Human Motion Prediction Ye Yuan, Kris Kitani ECCV, 2020 project page \| arXiv \| talk \| summary \| video \| code
	Efficient Non-Line-of-Sight Imaging from Transient Sinograms Mariko Isogawa, Dorian Yao Chan, Ye Yuan, Kris Kitani, Matthew O'Toole ECCV, 2020 project page \| arXiv \| summary \| video
	Diverse Trajectory Forecasting with Determinantal Point Processes Ye Yuan, Kris Kitani ICLR, 2020 arXiv \| openreview \| video
	Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation Mariko Isogawa, Ye Yuan, Matthew O'Toole, Kris Kitani CVPR, 2020 project page \| arXiv \| video \| code
	Generative Hybrid Representations for Activity Forecasting with No-Regret Learning Jiaqi Guan, Ye Yuan, Kris Kitani, Nick Rhinehart CVPR, 2020 (Oral Presentation - Top 5.7%) arXiv \| data
	Back-Hand-Pose: 3D Hand Pose Estimation for a Wrist-worn Camera via Dorsum Deformation Network Erwin Wu, Ye Yuan, Hui-Shyong Yeo, Aaron Quigley, Hideki Koike, Kris Kitani ACM Symposium on User Interface Software and Technology (UIST), 2020 paper \| video
	MonoEye: Multimodal Human Motion Capture System Using A Single Ultra-Wide Fisheye Camera Dong-Hyun Hwang, Kohei Aso, Ye Yuan, Kris Kitani, Hideki Koike ACM Symposium on User Interface Software and Technology (UIST), 2020 paper \| video
	Ego-Pose Estimation and Forecasting as Real-Time PD Control Ye Yuan, Kris Kitani ICCV, 2019 project page \| arXiv \| video \| code \| data
	3D Ego-Pose Estimation via Imitation Learning Ye Yuan, Kris Kitani ECCV, 2018 paper \| video
	Computational Design of Transformables Ye Yuan, Changxi Zheng, Stelian Coros ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA), 2018 paper \| video
	Computational Abstractions for Interactive Design of Robotic Devices Ruta Desai, Ye Yuan, Stelian Coros ICRA, 2017 paper \| video
	Continuous Optimization of Interior Carving in 3D Fabrication Yue Xie, Ye Yuan, Xiang Chen, Changxi Zheng, Kun Zhou Frontiers of Computer Science, 2017 paper

Service

Conference Reviewer	NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV, AAAI, ICRA, SIGGRAPH, Eurographics
Journal Reviewer	JMLR, TMLR, TPAMI, TIP, RA-L

Template adapted from this awesome website.