Zhiheng Liu

I am a master candidate at the University of Science and Technology of China (USTC), supervised by Yang Cao.

My research interests lie in AIGC, currently focusing on the generation and editing of video and 3D vision.

I am always open to research discussions and collaborations, feel free to get in touch!

Google Scholar  /  Github

profile photo

I weighed about 70 kilograms when this photo was taken, but now I'm at about 85 kilograms and currently on a diet. :)

News
  • [7. 2024] LivePhoto accepted to ECCV 2024.
  • [5. 2024] CCM accepted to ICML 2024.
  • [4. 2024] We release InFusion for 3D inpainting via diffusion prior.
  • [3. 2024] DreamVideo accepted to CVPR 2024.
  • [1. 2024] DreamClean accepted to ICLR 2024.
  • [12. 2023] This page is online. Discussions and collaborations are welcome.
Selected Publications

(*: Equal contribution)

InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior
Zhiheng Liu*, Hao Ouyang*, Qiuyu Wang, Ka Leong Cheng, Jie Xiao, Kai Zhu, Nan Xue, Yu Liu, Yujun Shen, Yang Cao
arxiv, 2024
pdf/ page/ code

We present an image-conditioned depth inpainting model, which uses the diffusion prior to inpaint 3D Gaussians and has very good geometric and texture consistency.

LivePhoto: Real Image Animation with Text-guided Motion Control
Xi Chen, Zhiheng Liu, Mengting Chen, Yutong Feng, Yu Liu, Yujun Shen, Hengshuang Zhao
ECCV, 2024
pdf/ page

We present LivePhoto, a real image animation method with text control. Different from previous works, LivePhoto truely listens to the text instructions and well preserves the object-ID.

Cones 2: Customizable Image Synthesis with Multiple Subjects
Zhiheng Liu*, Yifei Zhang*, Yujun Shen, Kecheng Zheng, Kai Zhu, Ruili Feng, Yu Liu, Deli Zhao, Jingren Zhou, Yang Cao
NeurIPS, 2023
pdf / page

Cones 2 uses a simple yet effective representation to register a subject. The storage space required for each subject is approximately 5 KB. Moreover, Cones 2 allows for the flexible composition of various subjects without any model tuning.

Cones: Concept Neurons in Diffusion Models for Customized Generation
Zhiheng Liu*, Ruili Feng*, Kai Zhu, Yifei Zhang, Kecheng Zheng, Yu Liu, Deli Zhao, Jingren Zhou, Yang Cao
ICML, 2023 Oral
pdf / code

We explore the subject-specific concept neurons in a pre-trained text-to-image diffusion model. Concatenating multiple clusters of concept neurons representing different persons, objects, and backgrounds can flexibly generate all related concepts in a single image.


Design and source code from Jon Barron's website