Research

I develop a research program on human-like perception and learning through structure, efficiency, and continual adaptation rather than brute-force scale. I am on the job market this year for research roles.

Across projects, a unifying mechanism is analysis by synthesis: framing perception and learning as inverse problems where models infer latent structure by constructing and testing generative explanations.

My goal as an independent PI is to build models that connect mechanistic insight to practical systems for robust, reusable intelligence.

Layer 1: Human-like perception (Perceptual Taxonomy)

Problem: Models recognize objects but fail to reason over hierarchy, attributes, and affordances.

Key idea: A cognitively grounded taxonomy and evaluation of scene reasoning.

Outcome: Diagnostic signals that reveal where perception breaks and how structure helps.

Layer 2: Structured 3D semantics & parts (Name That Part)

Problem: Parts are segmented without semantics or named without structure.

Key idea: Joint part segmentation and naming as set alignment with scalable annotation.

Outcome: Reusable, consistent part semantics that scale under constrained data.

Layer 3: Analysis by synthesis & world reconstruction

Problem: Perception lacks grounded 3D world models for reasoning and synthesis.

Key idea: Analysis by synthesis pipelines that reconstruct structured 3D worlds.

Outcome: World models that support part-level reasoning and compositional inference.

Layer 4: Efficient & strict continual learning (UWSH, EigenLoRAx, Share)

Problem: Continual adaptation is compute-heavy and prone to forgetting.

Key idea: Shared low-dimensional update geometry for reuse, merging, and strict continual learning.

Outcome: Sustainable adaptation that scales beyond privileged compute and data.

Selected Contributions and Research Ideas

  1. Universal weight geometry / shared subspaces. Studied and characterized shared low-dimensional structure across trained neural models.
  2. Named 3D parts as set alignment. Proposed a scalable approach for jointly segmenting and naming 3D parts.
  3. Hierarchical perceptual diagnostics. Developed a framework for evaluating compositional and structured visual reasoning.
  4. Catastrophic remembering + strict continual learning. Explored dynamics of remembering/forgetting and stricter continual-learning protocols.
  5. Source-free, image-only UDA for 3D pose (3DUDA). Introduced a source-free adaptation setting for 3D object pose estimation.
  6. Adapter recycling / principal subspaces. Reused adapters to estimate principal subspaces for efficient adaptation and reuse.
  7. Shared subspace continual fine-tuning (Share). Framed continual updates as shared-subspace fine-tuning for efficiency and stability.

Research Roadmap

A living view of completed recent projects, active directions, and the longer-term research program.

Completed recent works

  • Completed: Strict continual learning methods for large language models.
  • Completed: Reliable evaluation metrics for 3D reasoning systems.
  • Completed: 3D animal fur reconstruction and shape modelling.

Next directions

  • Efficient 3D primitive representations.
  • Probabilistic and semi-supervised semantic 3D part segmentation.
  • Part-based vision-language-action models.
  • LLM/VLM model merging and transfer.

Long-term direction

  • General theory of deep learning and scaling behavior.
  • Synthetic internal 3D worlds for part-level causal and physics reasoning.
  • Explore super-Turing perspectives (theoretical).