CVPR 2024 Papers — Page 16
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2716 papers
MGMap: Mask-Guided Learning for Online Vectorized HD Map Construction
Xiaolu Liu (Zhejiang University), Jianke Zhu (Zhejiang University)
Object DetectionSegmentationAutonomous DrivingTransformerSimultaneous Localization and MappingImagePoint Cloud
🎯 What it does: This paper studies a mask-guided online high-definition map vectorization method called MGMap, aimed at accurately locating road features and achieving real-time generation.
MICap: A Unified Model for Identity-Aware Movie Descriptions
Haran Raajesh (International Institute of Information Technology Hyderabad), Makarand Tapaswi (International Institute of Information Technology Hyderabad)
GenerationTransformerVideoTextMultimodality
🎯 What it does: This paper proposes a single-stage encoding-decoding model called MICap, designed to generate identity-aware descriptions or fill-in-the-blank (FITB) tasks on video collections.
MicroCinema: A Divide-and-Conquer Approach for Text-to-Video Generation
Yanhui Wang (University of Science and Technology of China), Baining Guo (Microsoft Research Asia)
GenerationData SynthesisDiffusion modelVideoText
🎯 What it does: This paper proposes a two-stage text-to-video generation framework—first generating keyframe images from text, and then generating video conditioned on the image and text using fine-tuned Stable Diffusion.
MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections
Mude Hui (University of California), Yuyin Zhou (University of California)
RestorationSegmentationDiffusion modelImage
🎯 What it does: MicroDiffusion is proposed, which combines INR and DDPM to recover high-quality 3D micro volumes from limited 2D mappings.
MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis
Dewei Zhou (Zhejiang University), Yi Yang (Zhejiang University)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper proposes the Multi-Instance Generation (MIG) task and designs the MIGC controller, which can precisely control the position, attributes, and quantity of multiple instances within a single image.
MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Grounding
Chun-Peng Chang (DFKI), Didier Stricker (DFKI)
RecognitionObject DetectionTransformerMultimodalityPoint CloudBenchmark
🎯 What it does: The MiKASA Transformer is proposed for 3D visual localization tasks, achieving an end-to-end trained multimodal fusion model that can simultaneously process semantic and spatial information.
MimicDiffusion: Purifying Adversarial Perturbation via Mimicking Clean Diffusion Model
Kaiyu Song (Sun Yat-sen University), Jian Yin (Sun Yat-sen University)
GenerationAdversarial AttackDiffusion modelImage
🎯 What it does: A new adversarial perturbation purification method based on diffusion models, called MimicDiffusion, is proposed to eliminate the impact of adversarial perturbations by mimicking the generation trajectory of undisturbed inputs.
Mind Artist: Creating Artistic Snapshots with Human Thought
Jiaxuan Chen, Gang Pan
GenerationData SynthesisGraph Neural NetworkTransformerDiffusion modelImageMultimodalityMagnetic Resonance Imaging
🎯 What it does: A dual-stream neural decoding framework named Mind Artist (MindArt) is proposed, which can directly reconstruct visual images from non-invasive fMRI signals and achieve stylized image generation without additional optimization.
Mind Marginal Non-Crack Regions: Clustering-Inspired Representation Learning for Crack Segmentation
Zhuangzhuang Chen (Shenzhen University), Jianqiang Li (Shenzhen University)
SegmentationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: A two-stage clustering heuristic representation learning framework (CIRL) is proposed, which first locates fuzzy regions by learning the predictive variance of marginal fuzzy areas, and then employs unsupervised clustering learning on these regions to enhance crack segmentation accuracy.
Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimation
Lior Talker, Michael Dinerstein
Depth EstimationConvolutional Neural NetworkPoint Cloud
🎯 What it does: Incorporate depth edge supervision into the sparse LIDAR supervised monocular depth estimation model to improve edge localization accuracy.
MindBridge: A Cross-Subject Brain Decoding Framework
Shizun Wang (National University of Singapore), Xinchao Wang (National University of Singapore)
GenerationData SynthesisDomain AdaptationDiffusion modelContrastive LearningImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: The MindBridge framework is proposed to achieve single-model cross-subject brain signal decoding, capable of reconstructing natural images from the fMRI of multiple subjects and supporting efficient adaptation for new subjects.
Minimal Perspective Autocalibration
Andrea Porfiri Dal Cin (Politecnico di Milano), Tomas Pajdla (CIIRC CTU Prague)
Depth EstimationOptimizationImage
🎯 What it does: A new minimal self-calibration method based on deep constraints is proposed, along with a complete classification of minimal problems and corresponding solvers.
Mining Supervision for Dynamic Regions in Self-Supervised Monocular Depth Estimation
Hoang Chuong Nguyen (Australian National University), Miaomiao Liu (NVIDIA)
Depth EstimationAutonomous DrivingOptical FlowImage
🎯 What it does: A self-supervised depth estimation framework is proposed, which generates scale-consistent pseudo-depth labels for dynamic scenes by separating static and dynamic regions.
Mip-Splatting: Alias-free 3D Gaussian Splatting
Zehao Yu (University of Tuebingen), Andreas Geiger (University of Tuebingen)
RestorationGenerationData SynthesisGaussian SplattingImage
🎯 What it does: The Mip-Splatting method is proposed, achieving real-time viewpoint synthesis without aliasing at arbitrary scales based on 3D Gaussian Splatting.
MirageRoom: 3D Scene Segmentation with 2D Pre-trained Models by Mirage Projection
Haowen Sun (Tsinghua University), Jiwen Lu (Tsinghua University)
SegmentationConvolutional Neural NetworkTransformerPoint Cloud
🎯 What it does: A 3D point cloud segmentation framework called MirageRoom is proposed, which enhances the coverage of projected images by distorting projection rays under heterogeneous media, combined with a pre-trained 2D model to achieve fine semantic segmentation.
Mirasol3B: A Multimodal Autoregressive Model for Time-Aligned and Contextual Modalities
AJ Piergiovanni (Google DeepMind), Anelia Angelova (Google DeepMind)
TransformerVideoTextMultimodality
🎯 What it does: A self-regressive model is proposed that splits multimodal learning into time-synchronized and non-synchronized parts, using a Combiner to jointly learn audio and video features and handle long videos.
Misalignment-Robust Frequency Distribution Loss for Image Transformation
Zhangkai Ni (Tongji University), Lin Ma (Meituan)
Image TranslationRestorationSuper ResolutionContrastive LearningImage
🎯 What it does: A frequency distribution loss (FDL) is proposed for image transformation tasks, achieving better image restoration and style transfer without aligned training data.
Mitigating Motion Blur in Neural Radiance Fields with Events and Frames
Marco Cannici (University of Zurich), Davide Scaramuzza (University of Zurich)
RestorationNeural Radiance FieldImage
🎯 What it does: Combining fuzzy images with event camera data, Ev-DeblurNeRF is proposed to restore clear NeRF scenes.
Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning
Zihua Zhao (Shanghai Jiao Tong University), Yanfeng Wang (Shanghai Jiao Tong University)
RetrievalContrastive LearningMultimodality
🎯 What it does: This study investigates the cross-modal retrieval task in the presence of noisy correspondences and proposes the Geometrical Structure Consistency (GSC) framework, which identifies and corrects noisy correspondences by simultaneously maintaining the consistency of both cross-modal and intra-modal geometric structures.
Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange
Yanhao Wu (Xi'an Jiaotong University), Mathieu Salzmann (École Polytechnique Fédérale de Lausanne)
SegmentationDomain AdaptationRepresentation LearningConvolutional Neural NetworkContrastive LearningPoint Cloud
🎯 What it does: This paper proposes a self-supervised learning framework based on object swapping, OESSL, to break the object dependency in indoor point cloud scenes and enhance feature robustness.
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
Sicong Leng (DAMO Academy), Lidong Bing (DAMO Academy)
Object DetectionGenerationTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: A visual contrast decoding (VCD) method is proposed, which suppresses object hallucinations in large visual-language models by contrasting the outputs of the original image and the distorted image with added Gaussian noise.
Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices
Huancheng Chen (University of Texas at Austin), Haris Vikalo (University of Texas at Austin)
Federated LearningSupervised Fine-TuningImage
🎯 What it does: This paper proposes the FedMPQ framework, which uses mixed precision quantization to enhance model performance in federated learning with heterogeneous resources.
MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning
Zhe Li (Huazhong University of Science and Technology), Stan Z. Li (Westlake University)
Representation LearningAdversarial AttackContrastive LearningImageTextMultimodalityBiomedical Data
🎯 What it does: A multi-modal pre-training based medical visual representation framework MLIP is proposed, which enhances transferable visual features by utilizing cross-modal alignment of medical images and reports.
MLP Can Be A Good Transformer Learner
Sihao Lin (RMIT University), Xiaojun Chang (University of Technology Sydney)
RecognitionObject DetectionSegmentationComputational EfficiencyTransformerGaussian SplattingImage
🎯 What it does: By analyzing the entropy of the attention layer and the subsequent MLP layer, this paper proposes to gradually sparsify low-entropy, low-interaction attention layers into identity mappings, and after fusing with residuals, directly input them into the following MLP, ultimately resulting in a Transformer block that contains only MLP, reducing parameters and computational load.
MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
Chaoyi Zhang (Microsoft), Lijuan Wang (Microsoft)
GenerationRetrievalTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoMultimodalityAudio
🎯 What it does: A training-agnostic, GPT-4-based MM-Narrator framework is proposed, utilizing multimodal perception, short-term/long-term memory, and multimodal contextual learning to achieve automatic audio description generation for long-duration videos.
MMA-Diffusion: MultiModal Attack on Diffusion Models
Yijun Yang (Chinese University of Hong Kong), Qiang Xu (Chinese University of Hong Kong)
GenerationAdversarial AttackTransformerPrompt EngineeringDiffusion modelImageTextMultimodality
🎯 What it does: A multi-modal attack framework called MMA-Diffusion is proposed and implemented, capable of attacking diffusion models in both text and image modalities, bypassing prompt filtering and post-processing security checks to generate high-quality NSFW content.
MMA: Multi-Modal Adapter for Vision-Language Models
Lingxiao Yang (Sun Yat-sen University), Xiaohua Xie (Sun Yat-sen University)
ClassificationDomain AdaptationTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: A multi-modal adapter (MMA) is proposed for efficient fine-tuning on few-shot generalization tasks using pre-trained vision-language models (such as CLIP).
MMCert: Provable Defense against Adversarial Attacks to Multi-modal Models
Yanting Wang (Pennsylvania State University), Jinyuan Jia (Pennsylvania State University)
RecognitionSegmentationAdversarial AttackMultimodalityAudio
🎯 What it does: This paper proposes MMCert, a provably robust defense framework that aggregates predictions through random subsampling of multimodal inputs to defend against l0 attacks on multimodal models.
MMM: Generative Masked Motion Model
Ekkasit Pinyoanuntapong (University of North Carolina at Charlotte), Chen Chen (University of Central Florida)
GenerationData SynthesisTransformerVideoText
🎯 What it does: A text-to-motion generation framework based on the Conditional Masked Motion Model (MMM) is proposed, which first quantizes 3D motion into discrete tokens using a motion tokenizer, and then generates motion sequences in parallel through a conditional masked Transformer.
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Xiang Yue (Ohio State University), Wenhu Chen (University of Waterloo)
TransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: A benchmark called MMMU has been constructed, covering 30 disciplines and 11.5K multimodal questions, to evaluate the multimodal understanding and reasoning capabilities of expert-level AI.
MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
Jielin Qiu (Carnegie Mellon University), Lijuan Wang (Microsoft Azure AI)
SegmentationGenerationTransformerVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: A large-scale, multi-modal (video + text) dataset for summary and thumbnail generation, called MMSum, has been proposed, along with a benchmark model for multi-modal summarization.
MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors
He Zhang (Beihang University), Xun Cao (Nanjing University)
Pose EstimationOptimizationRecurrent Neural NetworkVideoMultimodality
🎯 What it does: This paper proposes the MMVP multimodal motion capture dataset, which combines RGBD videos with foot pressure sensors to achieve precise foot contact labeling for a wide range of rapid movements. Based on this, a VP-MoCap visual-pressure version of SMPL fitting and a monocular video baseline method are introduced.
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training
Pavan Kumar Anasosalu Vasu (Apple), Oncel Tuzel (Apple)
RetrievalComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkTransformerReinforcement LearningContrastive LearningImageTextMultimodality
🎯 What it does: Designed and trained a series of efficient CLIP models for mobile devices, called MobileCLIP, and proposed a multimodal reinforcement training method to enhance the zero-shot classification and retrieval performance of small models.
Mocap Everyone Everywhere: Lightweight Motion Capture With Smartwatches and a Head-Mounted Camera
Jiye Lee (Seoul National University), Hanbyul Joo (Seoul National University)
Pose EstimationOptimizationTransformerSimultaneous Localization and MappingTime Series
🎯 What it does: A lightweight, low-cost full-body motion capture system based on two smartwatches and a head-mounted camera is proposed;
MoCha-Stereo: Motif Channel Attention Network for Stereo Matching
Ziyang Chen (Guizhou University), Jia Wu (Guizhou University)
RestorationDepth EstimationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes the MoCha-Stereo network, which introduces Motif Channel Attention (MCA) and Motif Channel Correlation Volume (MCCV) to recover lost edge information by mining repeated geometric motifs in feature channels, and further refines the disparity map at full resolution through Reconstruction Error Motif Penalty (REMP).
Modality-agnostic Domain Generalizable Medical Image Segmentation by Multi-Frequency in Multi-Scale Attention
Ju-Hyeon Nam (Inha University), Sang-Chul Lee (Inha University)
SegmentationDomain AdaptationConvolutional Neural NetworkImageBiomedical DataUltrasound
🎯 What it does: A medical image segmentation network named MADGNet has been designed and implemented, capable of achieving cross-modal (modality-agnostic) and cross-domain (domain generalizable) segmentation across different imaging modalities.
Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration
Tony C. W. Mok (Alibaba Group), Ling Zhang
OptimizationRepresentation LearningConvolutional Neural NetworkContrastive LearningMultimodalityBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: This paper proposes a modality-independent deep structural image representation learning method (MASR-Net), which learns deep structural image representations (DSIR) that possess contrast invariance and high discriminability for multimodal medical images through deep neighborhood self-similarity (DNS) and anatomy-based contrastive learning, thereby simplifying the multimodal registration problem into a unimodal one.
Modality-Collaborative Test-Time Adaptation for Action Recognition
Baochen Xiong (Chinese Academy of Sciences), Changsheng Xu (Chinese Academy of Sciences)
RecognitionDomain AdaptationOptical FlowVideoMultimodality
🎯 What it does: This paper studies a passive data multimodal video test-time adaptation method called MC-TTA, which aligns the pseudo source distribution and target distribution using a teacher/student memory bank.
ModaVerse: Efficiently Transforming Modalities with LLMs
Xinyu Wang, Qi Wu
GenerationData SynthesisComputational EfficiencyTransformerLarge Language ModelAgentic AIVision Language ModelDiffusion modelImageVideoTextMultimodalityAudio
🎯 What it does: A multimodal large language model named ModaVerse is proposed, capable of understanding and generating multimodal content such as images, videos, and audio.
MoDE: CLIP Data Experts via Clustering
Jiawei Ma (Meta), Hu Xu (Meta)
ClassificationRetrievalTransformerMixture of ExpertsContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes the MoDE (Mixture of Data Experts) framework, which divides the CLIP pre-training data into several semantically similar subsets through a two-step clustering process. It trains independent CLIP data experts for each subset and integrates the outputs of these experts based on the similarity between task metadata and clustering centers during inference, thereby improving zero-shot image classification and retrieval performance.
Model Adaptation for Time Constrained Embodied Control
Jaehyun Song (Sungkyunkwan University), Honguk Woo (Sungkyunkwan University)
Autonomous DrivingKnowledge DistillationRobotic IntelligenceMeta LearningReinforcement LearningImage
🎯 What it does: The MoDeC framework is proposed, achieving adaptive reasoning for time-constrained multi-task control models based on modular networks.
Model Inversion Robustness: Can Transfer Learning Help?
Sy-Tuyen Ho (Singapore University of Technology and Design), Ngai-Man Cheung (Stanford University)
ClassificationSafty and PrivacyAdversarial AttackSupervised Fine-TuningGenerative Adversarial NetworkImage
🎯 What it does: A model inversion defense method based on transfer learning, TL-DMI, is proposed, which restricts private data to fine-tuning only the last few layers of the model, thereby weakening the effectiveness of model inversion attacks.
Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use
Imad Eddine Toubal (Google Research), Tom Duerig (Google Research)
ClassificationKnowledge DistillationTransformerLarge Language ModelVision Language ModelImageTextChain-of-Thought
🎯 What it does: Proposes the Modeling Collaborator framework, which utilizes large language models and visual language models to automatically annotate and train subjective visual classification models through natural language interaction.
Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction
Guillaume Jaume (Mass General Brigham), Faisal Mahmood (CMU)
ClassificationExplainability and InterpretabilityTransformerMultimodalityBiomedical Data
🎯 What it does: This study investigates methods for predicting patient survival by combining whole slide images and transcriptomic data.
Modeling Multimodal Social Interactions: New Challenges and Baselines with Densely Aligned Representations
Sangmin Lee (University of Illinois Urbana-Champaign), James M. Rehg (University of Illinois Urbana-Champaign)
RecognitionObject DetectionPose EstimationTransformerLarge Language ModelVideoTextMultimodality
🎯 What it does: Three multi-party social interaction tasks are proposed (speaker recognition, pronoun resolution, mention prediction), along with multimodal data annotation and baseline models for social reasoning games like Werewolf;
Modular Blind Video Quality Assessment
Wen Wen (City University of Hong Kong), Kede Ma (City University of Hong Kong)
Convolutional Neural NetworkTransformerVideo
🎯 What it does: A modular blind video quality assessment model is proposed, enhancing the model's reusability through modular training.
MOHO: Learning Single-view Hand-held Object Reconstruction with Multi-view Occlusion-Aware Supervision
Chenyangguang Zhang, Xiangyang Ji
🎯 What it does: Unable to determine the specific research content of the paper
Molecular Data Programming: Towards Molecule Pseudo-labeling with Systematic Weak Supervision
Xin Juan (Jilin University), Xin Wang (Massachusetts Institute of Technology)
ClassificationDrug DiscoveryGraph Neural NetworkGraphTabular
🎯 What it does: A MDP framework is designed to generate molecular pseudo-labels using various weakly supervised label functions based on graph kernels, fingerprints, and topological features, and these probabilistic pseudo-labels are used to train graph neural networks for molecular property classification.
MoMask: Generative Masked Modeling of 3D Human Motions
Chuan Guo (University of Alberta), Li Cheng (University of Alberta)
GenerationData SynthesisPose EstimationTransformerLarge Language ModelGenerative Adversarial NetworkVideoText
🎯 What it does: This paper proposes MoMask, a text-driven 3D human motion generation framework based on residual vector quantization and generative mask Transformer, capable of generating high-quality and diverse motions from natural language descriptions.
MoML: Online Meta Adaptation for 3D Human Motion Prediction
Xiaoning Sun (Nanjing University of Science and Technology), Jianfeng Lu (Nanjing University of Science and Technology)
Pose EstimationOptimizationMeta LearningGraph Neural NetworkReinforcement LearningTime Series
🎯 What it does: An online meta-learning adaptive framework MoML is designed and implemented, enabling a 3D human motion prediction model to adjust parameters in real-time using recent prediction errors from streaming data, thereby improving prediction accuracy.
Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models
Zhang Li (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
RecognitionGenerationTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: The Monkey model is designed by segmenting high-resolution images into uniformly sized patches and using a shared ViT encoder + LoRA adapter, combined with a multi-level description generation method, to enhance the understanding and generation capabilities of LMM on high-resolution images.
MonoCD: Monocular 3D Object Detection with Complementary Depths
Longfei Yan (Huazhong University of Science and Technology), Yihua Tan (Huazhong University of Science and Technology)
Object DetectionDepth EstimationAutonomous DrivingConvolutional Neural NetworkImageBenchmark
🎯 What it does: This paper proposes a monocular 3D object detection method called MonoCD, which utilizes both global and local depth cues and designs complementary depth predictions through geometric relationships to improve depth estimation accuracy and overall detection performance.
Monocular Identity-Conditioned Facial Reflectance Reconstruction
Xingyu Ren (Shanghai Jiao Tong University), Xiaokang Yang (Shanghai Jiao Tong University)
RestorationGenerationTransformerAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: We propose a monocular identity-conditioned facial reflectance reconstruction framework, ID2Reflectance, which can directly estimate high-quality diffuse/specular lighting, roughness, and normal maps from ordinary RGB images, and render under arbitrary lighting conditions.
MonoDiff: Monocular 3D Object Detection and Pose Estimation with Diffusion Models
Yasiru Ranasinghe (Johns Hopkins University), Vishal M. Patel (Johns Hopkins University)
Object DetectionPose EstimationAutonomous DrivingDiffusion modelImageBenchmark
🎯 What it does: This paper proposes MonoDiff, which utilizes diffusion models to achieve 3D object detection and pose estimation from monocular images.
MonoHair: High-Fidelity Hair Modeling from a Monocular Video
Keyu Wu (Zhejiang University), Youyi Zheng (Zhejiang University)
GenerationOptimizationNeural Radiance FieldVideoMesh
🎯 What it does: The MonoHair framework is proposed, which first obtains rough geometry from monocular video using NeRF, then extracts fine external structures using Patch-based Multi-View Optimization (PMVO), subsequently inferring internal structures through rendered undirected wireframe graphs and an improved DeepMVSHair*, ultimately generating a complete 3D hair model.
MonoNPHM: Dynamic Head Reconstruction from Monocular Videos
Simon Giebenhain (Technical University of Munich), Matthias Nießner (Technical University of Munich)
Object TrackingGenerationPose EstimationNeural Radiance FieldImageVideo
🎯 What it does: Proposed Mono NPHM: a parametric head model based on neural fields that jointly represents geometry, appearance, and expression, achieving dynamic 3D head reconstruction in monocular RGB videos through inverse deformation and hyper-dimensional enhancement.
MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric
Haokun Lin (University of Chinese Academy of Sciences), Zhenan Sun (Chinese Academy of Sciences)
RetrievalCompressionKnowledge DistillationTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: A modular pruning error (MoPE) metric for compressing visual-language pre-trained models (CLIP) is proposed, and a unified unmasked structured pruning framework is constructed based on this metric, supporting efficient compression during both the pre-training and fine-tuning stages.
MoReVQA: Exploring Modular Reasoning Models for Video Question Answering
Juhong Min (Google Research), Cordelia Schmid (Google Research)
RetrievalExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoMultimodality
🎯 What it does: A multi-stage modular reasoning framework MoReVQA is proposed for video question answering, which includes three stages: event parsing, video localization, and reasoning, and utilizes external memory to record intermediate results.
Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation
Xiyi Chen, Siyu Tang
GenerationData SynthesisPose EstimationDiffusion modelImageMesh
🎯 What it does: This paper proposes a method that integrates 3D deformable models into a multi-view consistent diffusion framework, capable of generating high-quality, 3D consistent, and animatable human avatars (head or full body) from a single portrait.
MorpheuS: Neural Dynamic 360deg Surface Reconstruction from Monocular RGB-D Video
Hengyi Wang (University College London), Lourdes Agapito (University College London)
GenerationKnowledge DistillationDiffusion modelScore-based ModelVideo
🎯 What it does: MorpheuS achieves complete, photo-realistic 360° dynamic surface reconstruction from monocular RGB-D videos by mapping dynamic scenes to a hyper-dimensional unified canonical space and incorporating knowledge distillation from diffusion models.
Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology
Andrew H. Song (Mass General Brigham), Faisal Mahmood (Mass General Brigham)
SegmentationRepresentation LearningBiomedical Data
🎯 What it does: A Gaussian Mixture Model-based unsupervised whole slide representation learning framework, PANTHER, is proposed, which constructs whole slide vectors using a small number of morphological prototypes.
Mosaic-SDF for 3D Generative Models
Lior Yariv (Meta), Yaron Lipman (Weizmann Institute of Science)
GenerationData SynthesisTransformerFlow-based ModelPoint CloudMesh
🎯 What it does: This paper proposes a new 3D shape representation method called Mosaic-SDF, and based on it, trains a Flow Matching model to achieve high-quality 3D generation under class conditions and text conditions.
MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shading
Abdallah Dib (Ubisoft LaForge), Marc-André Carbonneau (Ubisoft LaForge)
RestorationGenerationGraph Neural NetworkAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: A method is proposed to reconstruct high-quality, re-illuminable 3D avatars from a single image, outputting 4K resolution geometry and various reflection attribute maps (diffuse, specular, ambient occlusion, subsurface scattering, normals).
MoST: Motion Style Transformer Between Diverse Action Contents
Boeun Kim (Korea Electronics Technology Institute), Jin Young Choi (University of Birmingham)
GenerationData SynthesisTransformerVideo
🎯 What it does: This paper proposes a motion style transfer framework named MoST, which can seamlessly transfer the style of source motion to target motion even when the source and target motion contents are inconsistent, without requiring any manually annotated style labels or post-processing steps.
MoST: Multi-Modality Scene Tokenization for Motion Prediction
Norman Mu (Waymo LLC), Yin Zhou (Waymo LLC)
Autonomous DrivingTransformerMultimodalityPoint CloudBenchmark
🎯 What it does: A method is proposed to decompose visual scenes into sparse scene elements and encode them into multimodal scene tokens using a pre-trained image model and LiDAR network for motion prediction.
Motion Blur Decomposition with Cross-shutter Guidance
Xiang Ji (University of Tokyo), Yinqiang Zheng (University of Tokyo)
RestorationSuper ResolutionConvolutional Neural NetworkTransformerOptical FlowImageVideo
🎯 What it does: A dual-view (fuzzy image + rolling shutter image) motion blur decomposition framework is proposed to address the temporal uncertainty in single-image blurry video reconstruction.
Motion Diversification Networks
Hee Jae Kim (Boston University), Eshed Ohn-Bar (Boston University)
GenerationPose EstimationTransformerAuto EncoderVideoMultimodality
🎯 What it does: A new 3D human motion generation framework called Motion Diversification Networks (MDN) is proposed, which generates diverse and realistic future motion sequences by deforming latent variables and combining motion primitives.
Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurring
Chengxu Liu (Xi'an Jiaotong University), Ming-Hsuan Yang (University of California, Merced)
RestorationConvolutional Neural NetworkOptical FlowImage
🎯 What it does: A motion-adaptive separable collaborative filter (MISC Filter) is proposed, which directly utilizes motion estimation information in the image space for adaptive deblurring of motion blur.
Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking
Wei Cao (Technical University of Munich), Jiapeng Tang (Technical University of Munich)
RestorationObject TrackingSegmentationTransformerDiffusion modelPoint CloudMesh
🎯 What it does: This paper proposes Motion2VecSets, a 4D latent vector set diffusion model designed to reconstruct dynamic surfaces and achieve motion tracking from sparse, noisy, or local point cloud sequences.
MotionEditor: Editing Video Motion via Content-Aware Diffusion
Shuyuan Tu (Fudan University), Yu-Gang Jiang (Fudan University)
GenerationData SynthesisTransformerDiffusion modelVideo
🎯 What it does: This paper presents MotionEditor, a video motion editing method based on diffusion models that can transfer motion from a reference video to a source video while keeping the appearance of the subject and background unchanged.
Move Anything with Layered Scene Diffusion
Jiawei Ren (Meta AI), Antoine Toisoul (Meta AI)
GenerationData SynthesisDiffusion modelImageText
🎯 What it does: This paper proposes SceneDiffusion, which utilizes a pre-trained text-to-image diffusion model to achieve controllable scene generation and spatial image editing (such as moving, scaling, cloning, and replacing) by optimizing hierarchical scene representations during the sampling process.
Move as You Say Interact as You Can: Language-guided Human Motion Generation with Scene Affordance
Zan Wang (Beijing Institute of Technology), Siyuan Huang
GenerationPose EstimationRobotic IntelligenceTransformerDiffusion modelMultimodalityPoint Cloud
🎯 What it does: A two-stage language-guided human action generation framework is proposed, utilizing 3D scene affordance as an intermediate representation to achieve more accurate scene localization and action synthesis.
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Enxin Song (Zhejiang University), Gaoang Wang (Shanghai Artificial Intelligence Laboratory)
RecognitionGenerationRetrievalTransformerLarge Language ModelVision Language ModelVideoTextBenchmark
🎯 What it does: We propose MovieChat, a long video understanding framework that combines visual models and LLMs, supporting question-answer interactions for long videos of over 10K frames.
MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception
Yiran Qin, Jing Shao
Robotic IntelligenceMultimodality
🎯 What it does: Unable to provide a summary, no paper content received.
mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration
Qinghao Ye (Alibaba Group), Fei Huang (Alibaba Group)
TransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: mPLUG-Owl2 is proposed, a multimodal large language model that integrates modality cooperation and modality adaptation modules, capable of unified performance in text and multimodal tasks;
MPOD123: One Image to 3D Content Generation Using Mask-enhanced Progressive Outline-to-Detail Optimization
Jimin Xu (Zhejiang University), Fei Wu (Zhejiang University)
GenerationOptimizationDiffusion modelImage
🎯 What it does: A two-stage progressive optimization framework MPOD123 is proposed to generate high-quality 3D content from a single image. It first optimizes the contour shape using a perspective-conditioned diffusion model, and then enhances local geometry and texture through detail appearance inpainting.
MR-VNet: Media Restoration using Volterra Networks
Siddharth Roheda (Samsung Research Institute), Loay Rashid (Samsung Research Institute)
RestorationConvolutional Neural NetworkImageVideo
🎯 What it does: A convolutional network MR-VNet based on the Volterra series is proposed for image and video restoration.
MRC-Net: 6-DoF Pose Estimation with MultiScale Residual Correlation
Yuelong Li (Amazon Inc), Sunil Hadap (Amazon Inc)
Pose EstimationConvolutional Neural NetworkImage
🎯 What it does: MRC-Net is proposed, a single-stage, RGB image-based 6-DoF pose estimation network that achieves high-accuracy estimation through a sequential structure of classification followed by regression.
MRFP: Learning Generalizable Semantic Segmentation from Sim-2-Real with Multi-Resolution Feature Perturbation
Sumanth Udupa (Indian Institute of Science), Suresh Sundaram (Indian Institute of Science)
SegmentationDomain AdaptationAutonomous DrivingConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: The MRFP (Multi-Resolution Feature Perturbation) module is proposed, which applies multi-resolution perturbations to the feature space during semantic segmentation training, significantly enhancing the generalization performance from simulation to real domains.
MRFS: Mutually Reinforcing Image Fusion and Segmentation
Hao Zhang (Wuhan University), Jiayi Ma (Tencent)
Image TranslationSegmentationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a coupled learning framework MRFS to simultaneously enhance the performance of infrared-visible image fusion and semantic segmentation.
MS-DETR: Efficient DETR Training with Mixed Supervision
Chuyang Zhao (Baidu), Jingdong Wang (Baidu)
Object DetectionSegmentationTransformerSupervised Fine-TuningImage
🎯 What it does: MS-DETR is proposed based on DETR, significantly improving the quality of candidate boxes and training efficiency by adding one-to-many supervision to the queries of the main decoder.
MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints
Pengfei Xie (Southeast University), Cewu Lu (Shanghai Jiao Tong University)
Object TrackingPose EstimationReinforcement LearningVideo
🎯 What it does: This paper proposes the integration of the musculoskeletal system with the learnable hand model MANO to construct MS-MANO, and employs BioPR for pose refinement in hand pose tracking.
MSU-4S - The Michigan State University Four Seasons Dataset
Daniel Kent (Michigan State University), Hayder Radha (Michigan State University)
Object DetectionDomain AdaptationAutonomous DrivingConvolutional Neural NetworkSimultaneous Localization and MappingMultimodalityPoint Cloud
🎯 What it does: The Michigan State University Four Seasons (MSU-4S) multimodal dataset has been proposed and released, containing multi-source data from cameras, LiDAR, millimeter-wave radar, GNSS, and CAN under different weather conditions (sunny, rainy, snowy, autumn) across the four seasons, with annotations for weather, time, location, and types of traffic participants.
MTLoRA: Low-Rank Adaptation Approach for Efficient Multi-Task Learning
Ahmed Agiza (Brown University), Sherief Reda (Brown University)
SegmentationOptimizationTransformerSupervised Fine-TuningImage
🎯 What it does: The MTLoRA framework is proposed, achieving parameter-efficient fine-tuning in multi-task learning models through task-agnostic (TA-LoRA) and task-specific (TS-LoRA) low-rank adaptation modules.
MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark
Sanghyun Woo, In So Kweon
RecognitionObject DetectionObject TrackingKnowledge DistillationContrastive LearningVideoMultimodalityBenchmark
🎯 What it does: This paper proposes MTMMC—a large-scale multimodal multi-camera tracking dataset consisting of 16 RGB + thermal imaging cameras, covering multi-layer buildings in real environments such as campuses and factories. Baseline models for multi-target multi-camera tracking, detection, and Re-ID tasks are constructed and evaluated based on this benchmark.
Mudslide: A Universal Nuclear Instance Segmentation Method
Jun Wang (Peking University)
Object DetectionSegmentationTransformerImage
🎯 What it does: A mudslide-based nuclear instance segmentation method called Mudslide is proposed.
MuGE: Multiple Granularity Edge Detection
Caixia Zhou (Beijing Jiaotong University), Haibin Ling (Stony Brook University)
TransformerContrastive LearningImage
🎯 What it does: The MuGE model is proposed, which can generate diverse edge maps based on edge detail granularity.
MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation
Petru-Daniel Tudosiu (Huawei Noah's Ark Lab), Sarah Parisot (Huawei Noah's Ark Lab)
Object DetectionSegmentationGenerationData SynthesisDiffusion modelImage
🎯 What it does: Developed an untrained image layering pipeline and generated the MuLAn dataset, which includes multi-layer RGBA decompositions and instance occlusion information for 44,860 images.
MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detection
Jakub Micorek (Graz University of Technology), Mateusz Kozinski (Graz University of Technology)
Anomaly DetectionScore-based ModelVideo
🎯 What it does: This paper proposes an end-to-end video anomaly detection method called MULDE, based on multi-scale log density estimation and denoising score matching.
Multi-agent Collaborative Perception via Motion-aware Robust Communication Network
Shixin Hong (Tsinghua University), You He (Tsinghua University)
Object DetectionAutonomous DrivingRecurrent Neural NetworkPoint Cloud
🎯 What it does: A motion-aware robust communication network named MRCNet is proposed, aimed at enhancing the robustness and accuracy of multi-agent collaborative perception in the presence of real-world noise such as pose noise, perception noise, and motion blur.
Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioning
Jaewoo Jeong (Korea Advanced Institute of Science and Technology), Kuk-Jin Yoon (Korea Advanced Institute of Science and Technology)
Pose EstimationTransformerTime Series
🎯 What it does: A two-level model T2P based on trajectory conditions is proposed for long-term multi-agent 3D human pose prediction.
Multi-Attribute Interactions Matter for 3D Visual Grounding
Can Xu (Nanjing University of Science and Technology), Jian Yang (Nanjing University of Science and Technology)
RecognitionObject DetectionTransformerMultimodalityPoint Cloud
🎯 What it does: This paper proposes a multi-attribute interactive Transformer that quantifies the causal effects of attributes on predictions through an attribute causal analysis module, and achieves fine-grained visual-language alignment for 3D visual localization tasks via an attribute interaction-driven cross-modal exchange fusion module.
Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers
Sanghyeok Lee (Korea University), Hyunwoo J. Kim (Korea University)
ClassificationComputational EfficiencyTransformerImage
🎯 What it does: This paper proposes a Multi-Criteria Token Fusion (MCTF) method that significantly reduces the number of tokens while maintaining the performance of visual Transformers.
Multi-Level Neural Scene Graphs for Dynamic Urban Environments
Tobias Fischer (ETH Zurich), Peter Kontschieder (Meta Reality Labs)
GenerationAutonomous DrivingGraph Neural NetworkNeural Radiance FieldImage
🎯 What it does: Proposes a multi-layer neural scene graph for reconstructing renderable radiance fields from images collected by multiple vehicles in dynamic urban environments.
Multi-Modal Hallucination Control by Visual Information Grounding
Alessandro Favero (EPFL), Stefano Soatto (Amazon)
GenerationOptimizationTransformerReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes an inference-time intervention method called Multi-Modal Mutual-Information Decoding (M3ID), aimed at reducing the phenomenon of 'hallucination' that occurs during the generation process of visual language models (VLM) by maximizing the mutual information between text generation and visual input. Furthermore, this method is combined with Direct Preference Optimization (DPO) to enhance the model's reliance on visual information.
Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizer
Zhen Zhao (East China Normal University), Yuan Xie (East China Normal University)
RecognitionTransformerLarge Language ModelImageTextMultimodality
🎯 What it does: A scene text recognition model named E STR is proposed, achieving rapid adaptation within a context learning framework without the need for additional fine-tuning.
Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception
Junwen He (Dalian University of Technology), Xuansong Xie (Alibaba Group)
SegmentationGenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityAudio
🎯 What it does: This paper presents AnyRef, a multimodal large language model capable of generating pixel-level masks and natural language descriptions from four types of multimodal prompts: text, bounding boxes, images, and audio.
Multi-modal Learning for Geospatial Vegetation Forecasting
Vitus Benson (Max Planck Institute for Biogeochemistry), Markus Reichstein (Max Planck Institute for Biogeochemistry)
TransformerMultimodalityTime SeriesAgriculture Related
🎯 What it does: The GreenEarthNet dataset and the Contextformer model are proposed for future predictions of high-resolution (20 m) NDVI of European vegetation.
Multi-Modal Proxy Learning Towards Personalized Visual Multiple Clustering
Jiawei Yao (University of Washington), Juhua Hu (Alibaba Group)
TransformerLarge Language ModelContrastive LearningMultimodality
🎯 What it does: A personalized multi-clustering method based on Multi-MaP (Multi-modal Proxy Learning) is proposed, which utilizes the CLIP encoder and GPT-4 to generate proxy words based on user keyword interests, and obtains the feature vectors required for clustering through similarity optimization.
Multi-Object Tracking in the Dark
Xinzhe Wang (Beijing Institute of Technology), Ying Fu (Beijing Institute of Technology)
Object DetectionObject TrackingVideo
🎯 What it does: A low-light multi-object tracking dataset LMOT has been established, and the LTrack method has been proposed, which performs detection and association directly on RAW low-light videos.