Zhiwei Deng (邓志伟)
Zhiwei Deng is a Research Scientist at Google DeepMind. He was a Postdoc at the Computer Science Department of Princeton University. He earned his PhD at Simon Fraser University in Computer Science.
He is generally interested in building intelligent learning agents with System 1&2. Recently, he is drawn by reparameterizing knowledge in neural networks through Input-Output space, and building perceptual world model using self-supervised representation learning.
Specifically, his works span across multiple representitive topics:
- The I/O space storage of information: Instead of storing the learned knowledge in the parameter space, pushing on the direction of I/O space representation (RememberThePast), and minimal set of data points for model training (ICONS, GTP).
- Visual self-supervised and generative models: Building the tokenizer for visual image via pure grouping operations (Perceptual Group Tokenizer) and Diffusion Models for inference time image editing (BoundaryDiffusion).
- Embodied Agents: Training topological navigation agents in vision-language world (Graph Planner), and planning agents in computer control (Anticipatory Agents).
Selected Publications and Preprints
For a full list of publications, please refer to this page.- ICONS: Influence Consensus for Vision-Language Data Selection
Xindi Wu, Mengzhou Xia, Rulin Shao, Zhiwei Deng, Pang Wei Koh, and Olga Russakovsky
arXiv 2024;
[arXiv]
- Influential Language Data Selection via Gradient Trajectory Pursuit
Zhiwei Deng, Tao Li, and Yang Li
arXiv 2024;
[arXiv]
- Devil's Advocate: Anticipatory Reflection for LLM Agents
Haoyu Wang, Tao Li, Zhiwei Deng, Dan Roth, and, Yang Li
EMNLP 2024;
[arXiv]
- Perceptual Group Tokenizer: Building Perception with Iterative Grouping
Zhiwei Deng, Ting Chen, and Yang Li
ICLR 2024;
[arXiv]
- What is Dataset Distillation Learning?
William Yang, Ye Zhu, Zhiwei Deng, and Olga Russakovsky
ICML 2024;
[arXiv] [Code]
- Vision-Language Dataset Distillation
Xindi Wu, Byron Zhang, Zhiwei Deng, and Olga Russakovsky
TMLR, ECCV Dataset Distillation Workshop (Best Paper Award) 2024;
Website | arXiv | TMLR Paper Site | Video (45 mins) | Slides(canva, pdf) | Poster | Code
- Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks
Zhiwei Deng and Olga Russakovsky
NeurIPS 2022;
[arXiv] [Code]
- Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
Zhiwei Deng, Karthik Narasimhan, and Olga Russakovsky
NeurIPS 2020;
[arXiv]
- BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps
Wang (Bill) Zhu*, Hexiang Hu*, Jiacheng Chen, Zhiwei Deng, Vihan Jain, Eugene Ie, and Fei Sha
ACL 2020 (Oral);
[arXiv]
- Probabilistic Neural Programmed Networks for Scene Generation
Zhiwei Deng, Jiacheng Chen, Yifang Fu, and Greg Mori
NeurIPS 2018 (Spotlight);
[arXiv]