Ye Yuan

I'm a Research Scientist in the Learning and Perception Team at NVIDIA Research. I received my Ph.D. in Robotics from Carnegie Mellon University in 2022, where I was advised by Prof. Kris Kitani. I also earned my M.S. in computer science at CMU in 2016, where I worked with Prof. Stelian Coros. I obtained my B.E. in computer science and technology from Zhejiang University in 2015. My research has been supported by the Qualcomm Innovation Fellowship and the NVIDIA Graduate Fellowship.

Email  |  CV  |  Google Scholar  |  Twitter  |  Github

profile photo
Sep 2022 One paper on embodied human pose estimation accepted to NeurIPS 2022!
May 2022 Joined NVIDIA Research as a Research Scientist!
Apr 2022 Defended my Ph.D. thesis Unified Simulation, Perception, and Generation of Human Behavior!
Mar 2022 One paper on global human mesh recovery accepted to CVPR 2022 with an Oral Presentation!
Jan 2022 One paper on efficient automatic agent design accepted to ICLR 2022 with an Oral Presentation!
Jan 2022 Invited Talk at MPI Perceiving Systems.
Sep 2021 One paper on kinematics-guided control accepted to NeurIPS 2021!
July 2021 One paper on multi-agent forecasting accepted to ICCV 2021!
Show more

My research lies at the intersection of computer vision, machine learning, and robotics. My long term goal is to empower Embodied Agents to Perceive, Predict, and Act in Physical environments. My current research interests span physics simulation, reinforcement learning, generative models, embodied agents, virtual humans, etc.

Embodied Scene-aware Human Pose Estimation
Zhengyi Luo, Shun Iwase, Ye Yuan, Kris Kitani
NeurIPS, 2022
project page | arXiv | video
Unified Simulation, Perception, and Generation of Human Behavior
Ye Yuan
Ph.D. Thesis, Robotics Institute, CMU, 2022
GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Ye Yuan, Umar Iqbal, Pavlo Molchanov, Kris Kitani, Jan Kautz
CVPR, 2022   (Oral Presentation - Top 4.2%)
project page | arXiv | video | code
Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris Kitani
ICLR, 2022   (Oral Presentation - Top 1.6%)
project page | arXiv | openreview | code
Online No-regret Model-Based Meta RL for Personalized Navigation
Yuda Song, Ye Yuan, Wen Sun, Kris Kitani
Learning for Dynamics & Control (L4DC), 2022
Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
Zhengyi Luo, Ryo Hachiuma, Ye Yuan, Kris Kitani
NeurIPS, 2021
project page | arXiv | video | code
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Ye Yuan, Xinshuo Weng, Yanglan Ou, Kris Kitani
ICCV, 2021
project page | arXiv | code
SimPoE: Simulated Character Control for 3D Human Pose Estimation
Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, Jason Saragih
CVPR, 2021   (Oral Presentation - Top 4.2%)
project page | arXiv | talk | video
PTP: Parallelized 3D Tracking and Prediction with Graph Neural Networks and Diversity Sampling
Xinshuo Weng*, Ye Yuan*, Kris Kitani   (*Equal Contribution)
RA-L and ICRA, 2021   (Best Student Paper Candidate < 2%)
project page | arXiv | code
Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
Ye Yuan, Kris Kitani
NeurIPS, 2020
project page | arXiv | video | code
DLow: Diversifying Latent Flows for Diverse Human Motion Prediction
Ye Yuan, Kris Kitani
ECCV, 2020
project page | arXiv | talk | summary | video | code
Efficient Non-Line-of-Sight Imaging from Transient Sinograms
Mariko Isogawa, Dorian Yao Chan, Ye Yuan, Kris Kitani, Matthew O'Toole
ECCV, 2020
project page | arXiv | summary | video
Diverse Trajectory Forecasting with Determinantal Point Processes
Ye Yuan, Kris Kitani
ICLR, 2020
arXiv | openreview | video
Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation
Mariko Isogawa, Ye Yuan, Matthew O'Toole, Kris Kitani
CVPR, 2020
project page | arXiv | video | code
Generative Hybrid Representations for Activity Forecasting with No-Regret Learning
Jiaqi Guan, Ye Yuan, Kris Kitani, Nick Rhinehart
CVPR, 2020   (Oral Presentation - Top 5.7%)
arXiv | data
Back-Hand-Pose: 3D Hand Pose Estimation for a Wrist-worn Camera via Dorsum Deformation Network
Erwin Wu, Ye Yuan, Hui-Shyong Yeo, Aaron Quigley, Hideki Koike, Kris Kitani
ACM Symposium on User Interface Software and Technology (UIST), 2020
paper | video
MonoEye: Multimodal Human Motion Capture System Using A Single Ultra-Wide Fisheye Camera
Dong-Hyun Hwang, Kohei Aso, Ye Yuan, Kris Kitani, Hideki Koike
ACM Symposium on User Interface Software and Technology (UIST), 2020
paper | video
Ego-Pose Estimation and Forecasting as Real-Time PD Control
Ye Yuan, Kris Kitani
ICCV, 2019
project page | arXiv | video | code | data
3D Ego-Pose Estimation via Imitation Learning
Ye Yuan, Kris Kitani
ECCV, 2018
paper | video
Computational Design of Transformables
Ye Yuan, Changxi Zheng, Stelian Coros
ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA), 2018
paper | video
Computational Abstractions for Interactive Design of Robotic Devices
Ruta Desai, Ye Yuan, Stelian Coros
ICRA, 2017
paper | video
Continuous Optimization of Interior Carving in 3D Fabrication
Yue Xie, Ye Yuan, Xiang Chen, Changxi Zheng, Kun Zhou
Frontiers of Computer Science, 2017
Conference Reviewer NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV, AAAI, ICRA, SIGGRAPH, Eurographics
Journal Reviewer JMLR, TMLR, TPAMI, TIP, RA-L

Template adapted from this awesome website.