Zhi Cao

I am a senior undergraduate in Computer Science at the University of Michigan and a research intern at Stanford University, fortunately advised by Dr. Huang Huang, Prof. Jiajun Wu, and Prof. Fei-Fei Li, and work closely with Dr. Yining Hong. At Michigan, I have been advised by Prof. Jason J. Corso and Prof. Alexander Rodríguez, and have worked closely with Yayuan Li and Dr. Tiange Luo.

My research focuses on:

  • Agentic robotics — robot agents that watch a human once, rehearse in simulation, and perform on a real robot, reasoning with transferable physical schemas.
  • Dexterous manipulation — beyond vision and proprioception, dexterous hands must anticipate and regulate interaction forces. I learn force-aware dexterity from large-scale human tactile data, for tool use, musical instruments, and precise lab procedures.
  • Human-to-robot transfer — transferring not just how humans move, but also how they perceive and think. From human consciousness to robot cognition, robots learn what people attend to, intend, and reason about while acting.

I am always happy to chat about research and potential collaborations — feel free to reach out!

profile photo

News

  • 2026.11: FACT received the Best Paper Award at the CoRL 2026 BiDexTool Workshop.
  • 2026.11: TrAct accepted as an Oral at the CoRL 2026 TAG Workshop.
  • 2026.10: Released EgoForce, the first open-source large-scale human tactile dataset.
  • 2026.02: BiMotion accepted to CVPR 2026 as a Highlight.
  • 2025.09: HANDI accepted to the ICCV 2025 HANDS workshop.
  • 2025.05: Joined the Stanford Vision and Learning Lab.
Selected Publications

* Equal contribution  ·  † Equal advising

FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity
Zhi Cao*, Yining Hong*, Jiaolong Yang, Yejin Choi, Li Fei-Fei, Jiajun Wu†, Huang Huang†
CoRL 2026 · BiDexTool · Best Paper Award
Website · Paper · Dataset (EgoForce)

FACT jointly models force, action, and visual tracks to transfer dexterous skills from human demonstrations to robot hands, enabling contact-rich tasks such as playing the harp.

TrAct: Bridging Robot Control and Visual Prediction with Visual Tracks
Zhi Cao*, Howard Ji*, Kevin Zhang*, Kuangzhi Ge, Li Fei-Fei, Jiajun Wu†, Huang Huang†
CoRL 2026 · TAG · Oral · Website · Paper

TrAct uses visual tracks as a shared interface between robot control and video prediction: it proposes action-track pairs, predicts their visual outcomes, and selects actions with a vision-language reward model.

T²Mem: Learning Test-Time Memory for Robotics
Yize Liu, Huang Huang, Yining Hong, Zijian Du, Zhi Cao, Li Fei-Fei, Jiajun Wu
arXiv 2026 · Website · Paper

T²Mem equips robot policies with a memory that is learned and updated at test time, letting them solve tasks that depend on what happened earlier in an episode.

BiMotion: B-spline Motion for Text-guided Dynamic 3D Character Generation
Miaowei Wang, Qingxuan Yan, Zhi Cao, Yayuan Li, Oisin Mac Aodha, Jason J. Corso, Amir Vaxman
CVPR 2026 · Highlight · Website · Paper

BiMotion represents character motion with B-spline curves, producing smooth and expressive text-guided dynamic 3D characters.

HANDI: Hand-Centric Text-and-Image Conditioned Video Generation
Yayuan Li*, Zhi Cao*, Jason J. Corso
ICCV 2025 · HANDS · Website · Paper

HANDI generates videos with detailed hand appearance and motion from an image and an action description, using an automatically inferred motion area and a hand refinement loss.

Overview of expert-guided counterfactual time series diffusion Counterfactual Time Series Diffusion Guided by Expert Models
Wenhao Mu, Zhi Cao, Mehmed Kerem Uludag, Alexander Rodríguez
Under review · Paper

A time-series diffusion framework for counterfactual prediction that incorporates guidance from imperfect expert models, staying robust when data are scarce.