Zhen Liu

Google Scholar | Twitter

 

Assistant professor @ CUHK-Shenzhen

 

Email : [my first name + last name]@cuhk.edu.cn


Bio

I am an assistant professor in the School of Data Science at CUHK-Shenzhen. I did my PhD in computer science at Mila and Université de Montréal with Liam Paull and Yoshua Bengio. During my PhD study, I visited at Max Planck Institute for Intelligent Systems, and worked with Michael J. Black and Bernhard Schölkopf. I received my M.S. and B.S. degrees in computer science with a minor in economics from Georgia Tech.

 

 

Students interested in Fall '26 admission are welcome to contact me (and potentially collaborate as an RA) in advance. We also welcone visiting students and RAs with strong background in either 3D generation or foundation models.

 

To apply for positions:

  • If you are affiliated with CUHK-Shenzhen. Shoot me an email with your resume and experiences.
  • If you are outside CUHK-Shenzhen. Please fill out the Google Form and then drop me an email if interested.
Due to the large volume of emails, I apologize that I am not able to reply to every email.

Research

I study representations and learning methods to develop models with a spatial, physical, and semantic understanding for re-creating and simulating our world. I draw on principles and theories from machine learning and related fields to discover simple and effective approaches that generalize well across diverse domains.

 

More specifically, I explore generative models (architecture, inference, finetuning, regularization, reasoning) and 3D representations (mesh, CAD model, etc.) for the following aspects:

  • Physics. How models understand and re-create real-world physics (shapes & motions)?
  • Assembly. How models understand and assemble complex 3D structures for us?
  • Semantics. How to achieve synergy of spatial and semantic understanding in foundation models?

 

And as a 3D vision researcher and a gamer, I always strive to make my research relevant to or applicable in games and movies (e.g., 3D content generation).

Recent Papers

Value Gradient Guidance for Flow Matching Alignment
Zhen Liu†, Tim Z. Xiao*, Carles Domingo-Enrich*, Weiyang Liu, Dinghuai Zhang†

Preprint

arXiv (Coming Soon) | TL;DR | bib 

First-order alignment / reward-finetuning method for flow matching models, tested on StableDiffusion 3.
Derived with HJB equation from optimal control theory.

Nabla-R2D3: Effective and Efficient 3D Diffusion Alignment with 2D Rewards
Qingming Liu*, Zhen Liu*†, Dinghuai Zhang, Kui Jia

Preprint

arXiv (Coming Soon) | TL;DR | bib 

Efficient, effective and robust alignment method for 3D-native diffusion models (e.g., 3DGS-based) with only 2D reward models.

Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling
Tim Z. Xiao, Johannes Zenn, Zhen Liu, Weiyang Liu, Robert Bamler, Bernhard Schölkopf

Preprint

arXiv | bib 

      @inproceedings{xiao2025flip,
        title={Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling},
        author={Tim Z. Xiao and Johannes Zenn and Zhen Liu and Weiyang Liu and Robert Bamler and Bernhard Schölkopf},
        booktitle={arXiv preprint arXiv:2506.09998},
        year={2025},
      }

ChatGarment: Garment Estimation, Generation and Editing via Large Language Models
Siyuan Bian, Chenghao Xu, Yuliang Xiu, Artur Grigorev, Zhen Liu, Cewu Lu, Michael J. Black, Yao Feng

Conference on Computer Vision and Pattern Recognition (CVPR), 2025

arXiv | project | bib 

      @inproceedings{bian2025chatgarment,
        title={ChatGarment: Garment Estimation, Generation and Editing via Large Language Models},
        author={Bian, Siyuan and Xu, Chenghao and Xiu, Yuliang and Grigorev, Artur and 
          Liu, Zhen and Lu, Cewu and Black, Michael J and Feng, Yao},
        journal={Conference on Computer Vision and Pattern Recognition (CVPR)},
        year={2025}
      }

Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets
Zhen Liu†, Tim Z. Xiao*, Weiyang Liu*, Yoshua Bengio, Dinghuai Zhang†

International Conference on Learning Representations (ICLR), 2025

arXiv | project | paper | bib 

      @inproceedings{liu2025nablagfn,
        title={Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets},
        author={Liu, Zhen and Xiao, Tim Z and Liu, Weiyang and Bengio, Yoshua and Zhang, Dinghuai},
        booktitle={International Conference on Learning Representations},
        year={2025},
      }

Publications

Show selected / all by date
(* for equal contribution and † for corresponding author)

Value Gradient Guidance for Flow Matching Alignment
Zhen Liu†, Tim Z. Xiao*, Carles Domingo-Enrich*, Weiyang Liu, Dinghuai Zhang†

Preprint

arXiv (Coming Soon) | TL;DR | bib 

First-order alignment / reward-finetuning method for flow matching models, tested on StableDiffusion 3.
Derived with HJB equation from optimal control theory.

Nabla-R2D3: Effective and Efficient 3D Diffusion Alignment with 2D Rewards
Qingming Liu*, Zhen Liu*†, Dinghuai Zhang, Kui Jia

Preprint

arXiv (Coming Soon) | TL;DR | bib 

Efficient, effective and robust alignment method for 3D-native diffusion models (e.g., 3DGS-based) with only 2D reward models.

Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling
Tim Z. Xiao, Johannes Zenn, Zhen Liu, Weiyang Liu, Robert Bamler, Bernhard Schölkopf

Preprint

arXiv | bib

ChatGarment: Garment Estimation, Generation and Editing via Large Language Models
Siyuan Bian, Chenghao Xu, Yuliang Xiu, Artur Grigorev, Zhen Liu, Cewu Lu, Michael J. Black, Yao Feng

Conference on Computer Vision and Pattern Recognition (CVPR), 2025

arXiv | project | bib 

      @article{bian2024chatgarment,
        title={ChatGarment: Garment Estimation, Generation and Editing via Large Language Models},
        author={Bian, Siyuan and Xu, Chenghao and Xiu, Yuliang and Grigorev, Artur and 
          Liu, Zhen and Lu, Cewu and Black, Michael J and Feng, Yao},
        journal={arXiv preprint arXiv:2412.17811},
        year={2024}
      }

Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets
Zhen Liu†, Tim Z. Xiao*, Weiyang Liu*, Yoshua Bengio, Dinghuai Zhang†

International Conference on Learning Representations (ICLR), 2025

arXiv | project | paper | bib 

      @inproceedings{liu2025nablagfn,
        title={Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets},
        author={Liu, Zhen and Xiao, Tim Z and Liu, Weiyang and Bengio, Yoshua and Zhang, Dinghuai},
        booktitle={International Conference on Learning Representations},
        year={2025},
      }

Can Large Language Models Understand Symbolic Graphics Programs?
Zeju Qiu*, Weiyang Liu*, Haiwen Feng*, Zhen Liu**, Tim Z. Xiao**, Katherine M. Collins**, Joshua B. Tenenbaum, Adrian Weller, Michael J. Black, Bernhard Schölkopf

International Conference on Learning Representations (ICLR), 2025 (Spotlight)

arXiv | code | project | bib 

      @article{qiu2025sgpbench,
        title={Can Large Language Models Understand Symbolic Graphics Programs?},
        author={Qiu, Zeju and Liu, Weiyang and Feng, Haiwen and Liu, Zhen and Xiao, Tim Z and Collins, Katherine M 
          and Tenenbaum, Joshua B and Weller, Adrian and Black, Michael J and Sch{\"o}lkopf, Bernhard},
          booktitle={International Conference on Learning Representations},
          year={2025},
      }

PuzzleAvatar: Assembling 3D Avatars from Personal Albums
Yuliang Xiu, Yufei Ye, Zhen Liu, Dimitrios Tzionas, Michael J. Black

ACM Transactions on Graphics (SIGGRAPH Asia), 2024

arXiv | video | bib 

        @article{xiu2024puzzleavatar,
          title={PuzzleAvatar: Assembling 3D Avatars from Personal Albums},
          author={Xiu, Yuliang and Ye, Yufei and Liu, Zhen and Tzionas, Dimitrios and Black, Michael J},
          journal={ACM Transactions on Graphics (TOG)},
          year={2024},
          publisher={ACM New York, NY, USA}
        }

Ghost on the Shell: An Expressive Representation of General 3D Shapes
Zhen Liu, Yao Feng*, Yuliang Xiu*, Weiyang Liu*, Liam Paull, Michael J. Black†, Bernhard Schölkopf†

International Conference on Learning Representations (ICLR), 2024 (Oral)

website | code | arXiv | pdf | bib 

  @InProceedings{Liu2024gshell,
        title = {Ghost on the Shell: An Expressive Representation of General 3D Shapes},
        author = {Liu, Zhen and Feng, Yao and Xiu, Yuliang and Liu, Weiyang and Paull, Liam and Black, Michael J. and Schölkopf, Bernhard},
        booktitle = {International Conference on Learning Representations},
        year = {2024}
      }

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
Weiyang Liu*, Zeju Qiu*, Yao Feng**, Yuliang Xiu**, Yuxuan Xue**, Longhui Yu**, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, Yandong Wen, Michael J. Black, Adrian Weller, Bernhard Schölkopf

International Conference on Learning Representations (ICLR), 2024

arXiv | code | project | pdf | bib

  @InProceedings{liu2023boft,
        author = {Liu, Weiyang and Qiu, Zeju and Feng, Yao and Xiu, Yuliang and Xue, Yuxuan and Yu, Longhui and Feng, Haiwen and Liu, Zhen 
          and Heo, Juyeon and Peng, Songyou and Wen, Yandong and Black, Michael J. and Weller, Adrian and Sch{\"o}lkopf, Bernhard},
        title = {Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization},
        booktitle = {International Conference on Learning Representations},
        year = {2024}
      }

Controlling Text-to-Image Diffusion by Orthogonal Finetuning
Zeju Qiu*, Weiyang Liu*, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, Bernhard Schölkopf

Neural Information Processing Systems (NeurIPS), 2023

arXiv | code | project | bib

  @InProceedings{Qiu2023OFT,
        title={Controlling Text-to-Image Diffusion by Orthogonal Finetuning},
        author={Qiu, Zeju and Liu, Weiyang and Feng, Haiwen and Xue, Yuxuan and Feng, Yao 
          and Liu, Zhen and Zhang, Dan and Weller, Adrian and Schölkopf, Bernhard},
          booktitle={NeurIPS},
          year={2023}
      }

Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods
Yuchen Lu, Zhen Liu, Aristide Baratin, Romain Laroche, Aaron Courville, Alessandro Sordoni

Transactions on Machine Learning Research (TMLR), 2023

openreview | bib

  @article{
        lu2023expresiveness,
        title={Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods},
        author={Yuchen Lu and Zhen Liu and Aristide Baratin and Romain Laroche and Aaron Courville and Alessandro Sordoni},
        journal={Transactions on Machine Learning Research},
        year={2023},
        url={https://openreview.net/forum?id=BxdrpnRHNh},
        }

MeshDiffusion: Score-based Generative 3D Mesh Modeling
Zhen Liu, Yao Feng, Michael J. Black, Derek Nowrouzezahrai, Liam Paull, Weiyang Liu

International Conference on Learning Representations (ICLR), 2023 (Notable-top-25%)

openreview | website | slides | code | arXiv | pdf | bib 

  
        @InProceedings{Liu2023MeshDiffusion,
          title={MeshDiffusion: Score-based Generative 3D Mesh Modeling},
          author={Zhen Liu and Yao Feng and Michael J. Black and Derek Nowrouzezahrai and Liam Paull and Weiyang Liu},
          booktitle={International Conference on Learning Representations},
          year={2023},
          url={https://openreview.net/forum?id=0cpM2ApF9p6}
          }

Iterative Teaching by Data Hallucination
Zeju Qiu*, Weiyang Liu*, Tim Z. Xiao, Zhen Liu, Umang Bhatt, Yucen Luo, Adrian Weller, Bernhard Schölkopf

International Conference on Artificial Intelligence and Statistics (AISTATS), 2023

arXiv

Continual Learning by Modeling Intra-Class Variation
Longhui Yu, Tianyang Hu, Lanqing Hong, Zhen Liu, Adrian Weller, Weiyang Liu

Transactions on Machine Learning Research (TMLR), 2023

arXiv | OpenReview

Structural Causal 3D Reconstruction
Weiyang Liu*, Zhen Liu*, Liam Paull, Adrian Weller, Bernhard Schölkopf

European Conference on Computer Vision (ECCV), 2022

arXiv

Generative Flow Networks for Discrete Probabilistic Modeling
Dinghuai Zhang, Nikolay Malkin, Zhen Liu, Alexandra Volokhova, Aaron Courville, Yoshua Bengio

International Conference on Machine Learning (ICML), 2022

arXiv

Iterative Teaching by Label Synthesis
Weiyang Liu*, Zhen Liu*, Hanchen Wang*, Liam Paull, Bernhard Schölkopf, Adrian Weller.

Neural Information Processing Systems (NeurIPS), 2021 (Spotlight)

arXiv

A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning
Mingde Zhao*, Zhen Liu*, Sitao Luan*, Shuyuan Zhang*, Doina Precup, Yoshua Bengio.

Neural Information Processing Systems (NeurIPS), 2021

arXiv | TL;DR | project | code 

 
        We show a consciousness-inspired bottleneck (i.e. agent attending to limited environment entities at a time) enables 
        model-based RL agents with out-of-distribution generalization capability.

Orthogonal Over-parametrized Training
Weiyang Liu*, Rongmei Lin*, Zhen Liu, James M Rehg, Liam Paull, Li Xiong, Adrian Weller, Le Song.

Conference on Computer Vision and Pattern Recognition (CVPR), 2021 (Oral)

arXiv | TL;DR | project | code | slides | talk 

 
        We parametrize convolution operator with a learnable rotation matrix multiplied by a fixed randomly-initialized matrix 
        so that a minimal hyperspherical energy of a network, a measure of neuron diversity and generalization, is guaranteed
        during the entire training process.
        - Generalization and optimization landscape are improved;
        - Also helpful for large category training.

Learning with Hyperspherical Uniformity
Weiyang Liu*, Rongmei Lin*, Zhen Liu*, Li Xiong, Bernhard Schölkopf, Adrian Weller.

International Conference on Artificial Intelligence and Statistics (AISTATS), 2021

arXiv

Neural similarity learning
Weiyang Liu*, Zhen Liu*, James M Rehg, Le Song.

Neural Information Processing Systems (NeurIPS), 2019

paper

Exponential Family Estimation via Adversarial Dynamics Embedding
Bo Dai*, Zhen Liu*, Hanjun Dai*, Niao He, Arthur Gretton, Le Song, Dale Schuurmans.

Neural Information Processing Systems (NeurIPS), 2019

arXiv | code

Coupled Variational Bayes via Optimization Embedding
Bo Dai, Hanjun Dai, Niao He, Weiyang Liu, Zhen Liu, Jianshu Chen, Lin Xiao, Le Song.

Neural Information Processing Systems (NeurIPS), 2018

paper

Learning towards Minimum Hyperspherical Energy
Weiyang Liu*, Rongmei Lin*, Zhen Liu*, Lixin Liu*, Zhiding Yu, Bo Dai, Le Song.

Neural Information Processing Systems (NeurIPS), 2018

arXiv | code

Decoupled Network
Weiyang Liu*, Zhen Liu*, Zhiding Yu, Bo Dai, Rongmei Lin, Yisen Wang, James Rehg, Le Song.

Conference on Computer Vision and Pattern Recognition (CVPR), 2018 (Spotlight)

pdf | code | slide

Towards Black-box Iterative Machine Teaching
Weiyang Liu*, Bo Dai*, Xingguo Li, Zhen Liu, James Rehg, Le Song.

International Conference on Machine Learning (ICML), 2018

arXiv | code

SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation
Bo Dai, Albert Shaw, Lihong Li, Lin Xiao, Niao He, Zhen Liu, Jianshu Chen, Le Song.

International Conference on Machine Learning (ICML), 2018

arXiv

Deep Forward and Inverse Perceptual Models for Tracking and Prediction
Alexander Lambert, Amirreza Shaban, Amit Raj, Zhen Liu, Byron Boots.

International Conference on Robotics and Automation (ICRA), 2018

arXiv

One Shot Learning for Semantic Segmentation
Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, Byron Boots.

The British Machine Vision Conference (BMVC), 2017

arXiv | code

Motion Planning with Graph-Based Trajectories and Gaussian Process Inference
Eric Huang, Mustafa Mukadam, Zhen Liu, Byron Boots.

International Conference on Robotics and Automation (ICRA), 2017

pdf | demo