Yining Hong picture

Yining Hong

University of California, Los Angeles
Department of Computer Science

About Me

I am currently a fourth-year PhD student in computer science at UCLA, co-advised by Prof. Ying Nian Wu and Prof. Kai-Wei Chang. I was under the supervision of Prof. Song-Chun Zhu from 2019 to 2021. I also interned at MIT-IBM Watson AI Lab, advised by Chuang Gan. Previously, I was an undergraduate in the Department of Electrical Engineering in Shanghai Jiao Tong University.

Research Highlights

I'm dedicated to building general-purpose embodied agents that could actively explore and interact with the 3D physical world, and perform common sense reasoning within the embodied environment. Specifically, I think the critical aspects of building such embodied agents reside in:



MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D World

Compositional VLM: Composing Visual Entities and Relationships in Large Language Models Via Communicative Decoding

Generative Neuro-Symbolic Visual Reasoning by Growing and Reusing Modules

3D-LLM: Injecting the 3D World into Large Language Models

3D Concept Learning and Reasoning from Multi-View Images

See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning

A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics

3D Concept Grounding on Neural Fields

Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction

PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning

VLGrammar: Grounded Grammar Induction of Vision and Language

Learning by Fixing: Solving Math Word Problems with Weak Supervision

SMART: A Situation Model for Algebra Story Problems via Attributed Grammar

Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks

A Competence-aware Curriculum for Visual Concepts Learning via Question Answering

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning

Academic Reader: An Interactive Question Answering System on Academic Literatures

Awards & Honors

Invited Talks


Outside of research, I'm a multi-instrumentalist. I play the piano, pipa (a Chinese instrument), and harpsichord in an orchestra; the pipe organ in a local church; the keyboard and guitar in a rock band. I also occasionally play the saxophone and melodica (highly underrated instrument!) in casual jazz jams. (Favorite music genres: metal, jazz, classic, rock. Favorite book.)

I've been playing the piano since I was five, and I was the president of the piano association in Shanghai Jiao Tong University. I can do fast sightreading on staff / lead sheet, and have rich recording / performing experience on piano and pipa.

Right now I'm really into improvisation and composition. I feel tons of music inside me and the urge to express it.