Ye Yuan

I'm a Research Scientist in the Learning and Perception Team at NVIDIA Research. I received my Ph.D. in Robotics from Carnegie Mellon University in 2022, where I was advised by Prof. Kris Kitani. I also earned my M.S. in computer science at CMU in 2016, where I worked with Prof. Stelian Coros. I obtained my B.E. in computer science and technology from Zhejiang University in 2015. My research has been supported by the Qualcomm Innovation Fellowship and the NVIDIA Graduate Fellowship.

Email | CV | Google Scholar | Twitter | Github

News

Mar	2023	One paper on learning digital tennis player from videos accepted to SIGGRAPH 2023.
Feb	2023	One paper on simulating pedestrian motions accepted to CVPR 2023.
Sep	2022	One paper on embodied human pose estimation accepted to NeurIPS 2022.
May	2022	Joined NVIDIA Research as a Research Scientist.
Apr	2022	Defended my Ph.D. thesis Unified Simulation, Perception, and Generation of Human Behavior.
Mar	2022	One paper on global human mesh recovery accepted to CVPR 2022 with an Oral Presentation.
Jan	2022	One paper on efficient automatic agent design accepted to ICLR 2022 with an Oral Presentation.
Jan	2022	Invited Talk at MPI Perceiving Systems.
Sep	2021	One paper on kinematics-guided control accepted to NeurIPS 2021.
July	2021	One paper on multi-agent forecasting accepted to ICCV 2021.
May	2021	Starting my internship at NVIDIA AI.
Apr	2021	Invited Talk at ETH Zurich, Computer Vision and Learning Group.
Apr	2021	Invited Talk at "Machine Learning and Optimal Control" class, University of Alabama.
Mar	2021	Received an outstanding reviewer award by ICLR 2021.
Feb	2021	One paper on physically-plausible human pose estimation accepted to CVRP 2021 with an Oral Presentation.
Feb	2021	One paper accepted to ICRA 2021.
Feb	2021	Invited Talk at 16th CSL student conference.
Feb	2021	Invited Talk at UIUC Robotics Seminar.
Dec	2020	Invited Talks at Qualcomm and Wayve.
Dec	2020	Honored to receive 2021 NVIDIA Graduate Fellowship.
Sep	2020	One paper on Residual Force Control accepted to NeurIPS 2020.
Aug	2020	Honored to receive 2020 Qualcomm Innovation Fellowship.
July	2020	Two papers accepted to ECCV 2020.
May	2020	Starting my internship at Facebook Reality Lab Pittsburgh.
Feb	2020	Two papers accepted to CVPR 2020, one with an Oral Presentation.
Dec	2019	Paper Diverse Trajectory Forecasting with Determinantal Point Processes accepted to ICLR 2020.
July	2019	Paper Ego-Pose Estimation and Forecasting as Real-Time PD Control accepted to ICCV 2019.
July	2018	Paper 3D Ego-Pose Estimation via Imitation Learning accepted to ECCV 2018.

Research

My research lies at the intersection of computer vision, machine learning, and robotics. My long term goal is to empower Embodied Agents to Perceive, Predict, and Act in Physical environments. My current research interests span physics simulation, reinforcement learning, generative models, embodied agents, digital humans, etc.

	PhysDiff: Physics-Guided Human Motion Diffusion Model Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz arXiv, 2022 project page \| arXiv \| video
	Learning Physically Simulated Tennis Players from Broadcast Videos Haotian Zhang, Ye Yuan, Viktor Makoviychuk, Yunrong Guo, Sanja Fidler, Xue Bin Peng, Kayvon Fatahalian SIGGRAPH, 2023 project page \| paper \| video
	Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Sanja Fidler, Or Litany CVPR, 2023 project page \| arXiv \| demo
	RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay, Bowen Wen, Ye Yuan, Stephen Tyree, Charles Loop, Alexander Schwing, Stan Birchfield ICRA, 2023 project page \| arXiv \| video \| data
	Embodied Scene-aware Human Pose Estimation Zhengyi Luo, Shun Iwase, Ye Yuan, Kris Kitani NeurIPS, 2022 project page \| arXiv \| video
	Unified Simulation, Perception, and Generation of Human Behavior Ye Yuan Ph.D. Thesis, Robotics Institute, CMU, 2022 arXiv
	GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras Ye Yuan, Umar Iqbal, Pavlo Molchanov, Kris Kitani, Jan Kautz CVPR, 2022 (Oral Presentation - Top 4.2%) project page \| arXiv \| video \| code
	Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris Kitani ICLR, 2022 (Oral Presentation - Top 1.6%) project page \| arXiv \| openreview \| code
	Online No-regret Model-Based Meta RL for Personalized Navigation Yuda Song, Ye Yuan, Wen Sun, Kris Kitani Learning for Dynamics & Control (L4DC), 2022 paper
	Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation Zhengyi Luo, Ryo Hachiuma, Ye Yuan, Kris Kitani NeurIPS, 2021 project page \| arXiv \| video \| code
	AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting Ye Yuan, Xinshuo Weng, Yanglan Ou, Kris Kitani ICCV, 2021 project page \| arXiv \| code
	SimPoE: Simulated Character Control for 3D Human Pose Estimation Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, Jason Saragih CVPR, 2021 (Oral Presentation - Top 4.2%) project page \| arXiv \| talk \| video
	PTP: Parallelized 3D Tracking and Prediction with Graph Neural Networks and Diversity Sampling Xinshuo Weng, Ye Yuan*, Kris Kitani (Equal Contribution) RA-L and ICRA, 2021 (Best Student Paper Candidate < 2%) project page \| arXiv \| code
	Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis Ye Yuan, Kris Kitani NeurIPS, 2020 project page \| arXiv \| video \| code
	DLow: Diversifying Latent Flows for Diverse Human Motion Prediction Ye Yuan, Kris Kitani ECCV, 2020 project page \| arXiv \| talk \| summary \| video \| code
	Efficient Non-Line-of-Sight Imaging from Transient Sinograms Mariko Isogawa, Dorian Yao Chan, Ye Yuan, Kris Kitani, Matthew O'Toole ECCV, 2020 project page \| arXiv \| summary \| video
	Diverse Trajectory Forecasting with Determinantal Point Processes Ye Yuan, Kris Kitani ICLR, 2020 arXiv \| openreview \| video
	Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation Mariko Isogawa, Ye Yuan, Matthew O'Toole, Kris Kitani CVPR, 2020 project page \| arXiv \| video \| code
	Generative Hybrid Representations for Activity Forecasting with No-Regret Learning Jiaqi Guan, Ye Yuan, Kris Kitani, Nick Rhinehart CVPR, 2020 (Oral Presentation - Top 5.7%) arXiv \| data
	Back-Hand-Pose: 3D Hand Pose Estimation for a Wrist-worn Camera via Dorsum Deformation Network Erwin Wu, Ye Yuan, Hui-Shyong Yeo, Aaron Quigley, Hideki Koike, Kris Kitani ACM Symposium on User Interface Software and Technology (UIST), 2020 paper \| video
	MonoEye: Multimodal Human Motion Capture System Using A Single Ultra-Wide Fisheye Camera Dong-Hyun Hwang, Kohei Aso, Ye Yuan, Kris Kitani, Hideki Koike ACM Symposium on User Interface Software and Technology (UIST), 2020 paper \| video
	Ego-Pose Estimation and Forecasting as Real-Time PD Control Ye Yuan, Kris Kitani ICCV, 2019 project page \| arXiv \| video \| code \| data
	3D Ego-Pose Estimation via Imitation Learning Ye Yuan, Kris Kitani ECCV, 2018 paper \| video
	Computational Design of Transformables Ye Yuan, Changxi Zheng, Stelian Coros ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA), 2018 paper \| video
	Computational Abstractions for Interactive Design of Robotic Devices Ruta Desai, Ye Yuan, Stelian Coros ICRA, 2017 paper \| video
	Continuous Optimization of Interior Carving in 3D Fabrication Yue Xie, Ye Yuan, Xiang Chen, Changxi Zheng, Kun Zhou Frontiers of Computer Science, 2017 paper

Service

Conference Reviewer	NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV, AAAI, ICRA, SIGGRAPH, Eurographics
Journal Reviewer	JMLR, TMLR, TPAMI, TIP, RA-L

Template adapted from this awesome website.