Not All Modalities Are Equal: Instruction-Aware Gating for Multimodal Videos
UniMVU fuses video, audio, depth, and other streams according to the instruction at hand, with instruction-conditioned gates within and across modalities.
About
Bonan Ding is a Computer Vision Ph.D. student at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), supervised by Prof. Fahad Khan and Prof. Salman Khan. His research spans multimodal video understanding, multimodal large language models, world models, video generation, and 3D perception.
Email: bonan.ding@mbzuai.ac.ae
Current: Research Intern at Meta Superintelligence Labs, working on audio-video generation in the Multimedia team · Aug. 10, 2026 – Present
News
- Joined Meta Superintelligence Labs as a Research Intern.
- Joined MBZUAI as a Ph.D. student in Computer Vision.
- SSLFusion appeared at AAAI 2025.
Selected Publications
Google ScholarSSLFusion: Scale and Space Aligned Latent Fusion Model for Multimodal 3D Object Detection
SSLFusion aligns scale and spatial representations between image features and point-cloud features for multimodal 3D detection.
C2BN: Cross-Modality and Cross-Scale Balance Network for Multi-Modal 3D Object Detection
C2BN balances modality importance, receptive fields, and multi-scale feature learning in Camera–LiDAR fusion.
Vision Foundation Model Driven Foreground-Aware Pseudo-LiDAR Generation for Monocular 3D Object DetectionarXiv preprint arXiv:2404.09431
Experience
Research Intern · Meta Superintelligence Labs
Working on audio-video generation in the Multimedia team.
Research Intern (Autonomous Driving Perception) · DiDi Voyager Autonomous Driving
Multimodal large language models for end-to-end autonomous driving.
Education
Ph.D. in Computer Vision · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
