🎯 What it does: A disassembly network (RUN) based on Robust Principal Component Analysis (RPCA) is proposed, which models the spatial downsampling error as a sparse Spatial Offset Component (SOC), thereby eliminating the need for approximating complex spatial downsampling matrices.
🎯 What it does: This paper reduces the distribution difference of intermediate features by incorporating optimal transport-based Max-Sliced Wasserstein distance regularization during the training process, allowing for the direct conversion of multiple layers into equivalent linear blocks and their simultaneous pruning in the post-training phase.
🎯 What it does: This paper proposes LaneDiffusion, which utilizes a diffusion model to generate lane centerline priors in the Bird's Eye View (BEV) feature layer, and then obtains high-quality lane centerline maps and topological structures through a decoder.
CodeSegmentationAutonomous DrivingTransformerVision Language ModelAuto EncoderImagePoint Cloud
🎯 What it does: Designed and implemented the LOcc framework, which generates dense fine-grained 3D language occupancy ground truth using a semantic transfer label pipeline, and trains an open vocabulary occupancy prediction network; at the same time, it modifies the prediction head of existing supervised occupancy models to make it compatible with both geometric and language heads.
Language-Driven Multi-Label Zero-Shot Learning with Semantic Granularity
Shouwen Wang (Huazhong University of Science and Technology), Zhigang Zeng (Medical University of Vienna)
CodeClassificationTransformerLarge Language ModelVision Language ModelContrastive LearningImageText
🎯 What it does: Utilizing large language models to generate text descriptions that include visual attributes, hierarchical relationships, and co-occurring scenes, and learning distinguishable reconstructed category names in the CLIP shared space to achieve multi-label zero-shot classification without image data;
🎯 What it does: The paper studies how using a larger learning rate (LR) in deep learning training can simultaneously enhance the model's robustness to spurious correlations and the network's compressibility.
Xinyu Yan (Tianjin University), Deng-Ping Fan (Nankai University)
CodeSegmentationDiffusion modelAuto EncoderImage
🎯 What it does: This paper proposes LawDIS, a controllable binary image segmentation method implemented using a stable diffusion model, supporting two modes: language prompts and window refinement.
Lay-Your-Scene: Natural Scene Layout Generation with Diffusion Transformers
Divyansh Srivastava (University of California San Diego), Zhuowen Tu (University of California San Diego)
CodeObject DetectionGenerationTransformerLarge Language ModelDiffusion modelImageText
🎯 What it does: This paper proposes an open vocabulary text-to-scene layout generation framework called Lay-Your-Scene, which is based on a lightweight open-source language model and a diffusion Transformer. It can automatically generate two-dimensional bounding box layouts that comply with spatial and counting constraints based on natural language prompts.
🎯 What it does: The LayerTracer framework is proposed to achieve hierarchical SVG generation and vectorization from text or images for cognitive alignment.
LazyMAR: Accelerating Masked Autoregressive Models via Feature Caching
Feihong Yan (Beijing Institute of Technology), Linfeng Zhang (Shanghai Jiaotong University)
CodeGenerationTransformerImage
🎯 What it does: This paper proposes LazyMAR, which accelerates the inference of the Masked Autoregressive (MAR) image generation model through feature caching technology.
Huaqiu Li (Tsinghua University), Xiangxiang Chu (Alibaba Group)
CodeRestorationLarge Language ModelDiffusion modelImage
🎯 What it does: A recursive posterior sampling framework based on latent diffusion models, LD-RPS, is proposed, which can achieve various image restoration tasks without the need for a training set and under zero-shot conditions.
🎯 What it does: We propose LeanVAE, an efficient video VAE for video compression and decoding in Latent Video Diffusion Models (LVDM), significantly reducing FLOPs while maintaining high quality;
🎯 What it does: A learnable fractional-order reaction-diffusion dynamics framework (LFRD) that combines physical models is proposed, which enhances the quality of Under-Display ToF depth maps through iterative optimization and validates its effectiveness in a broader range of depth recovery tasks.
Learnable Retrieval Enhanced Visual-Text Alignment and Fusion for Radiology Report Generation
Qin Zhou (East China University of Science and Technology), Sai Wu (Zhejiang University)
CodeGenerationRetrievalTransformerVision Language ModelImageTextMultimodalityElectronic Health RecordsRetrieval-Augmented Generation
🎯 What it does: A learnable retrieval-enhanced visual-text alignment and fusion framework (REVTAF) is proposed for generating radiology reports from lung X-ray images.
Learning Beyond Still Frames: Scaling Vision-Language Models with Video
Yiyuan Zhang (Chinese University of Hong Kong), Xiangyu Yue (Chinese University of Hong Kong)
CodeClassificationRetrievalTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: This paper presents LLaVA-Prime, which combines video pre-training to extend the visual language model of LLaVA, using Causal Hierarchical Aggregation (CHA) to efficiently process videos and enhance temporal understanding of vision and language.
🎯 What it does: A two-stage framework L2M is proposed, which elevates single-view 2D images to 3D space, utilizes multi-view synthesis and 3D Gaussian features to train a 3D perception encoder, and employs new view rendering with large-scale synthetic data to train a robust feature decoder for dense feature matching.
🎯 What it does: By constructing a dual-layer Human Gaussian Graph, the 3D Gaussians predicted from multiple frames in the video are associated with the SMPL mesh, enabling the generation of animatable human Gaussian representations during inference.
Learning Normals of Noisy Points by Local Gradient-Aware Surface Filtering
Qing Li (Southwest Jiaotong University), Yu-Shen Liu (Tsinghua University)
CodeRestorationPoint Cloud
🎯 What it does: This paper proposes an unsupervised point cloud normal estimation method based on local gradient-aware surface filtering, which can directly recover high-quality normal vectors from noisy point clouds and achieve surface reconstruction and denoising.
Learning Robust Stereo Matching in the Wild with Selective Mixture-of-Experts
Yun Wang (City University of Hong Kong), Junjie Hu (Chinese University of Hong Kong)
CodeDepth EstimationDomain AdaptationAutonomous DrivingTransformerMixture of ExpertsImage
🎯 What it does: This paper proposes SMoEStereo, a robust stereo matching framework that adapts to different scenes by introducing selective mixture of experts (MoE) and low-rank adaptation (LoRA) into the visual foundation model (VFM) along with a lightweight decision network.
🎯 What it does: This paper proposes an open vocabulary action recognition framework called Open-MeDe based on meta-learning, which significantly suppresses the static bias of CLIP through cross-batch meta-optimization and Gaussian self-ensemble, achieving stronger known-to-open generalization and debiasing from image to video.
CodeRecognitionDomain AdaptationSafty and PrivacyAuto EncoderGenerative Adversarial NetworkVideo
🎯 What it does: A privacy-preserving action recognition framework called GenPriv based on generative decoupled learning is proposed, achieving cross-domain transfer on labeled videos in the source domain and unlabeled videos in the target domain.
Less-to-More Generalization: Unlocking More Controllability by In-Context Generation
Shaojin Wu (ByteDance), Qian He (ByteDance)
CodeGenerationData SynthesisTransformerVision Language ModelDiffusion modelImageMultimodality
🎯 What it does: The UNO framework is proposed, achieving high consistency and controllability in image generation under single/multiple subject conditions through model-data co-evolution, evolutionary cross-modal alignment, and full rotation position embedding.
🎯 What it does: Proposes an attack method utilizing spatially invariant multi-scale and multi-position input transformations (SID) to enhance the transferability of adversarial samples against DNNs.
Leveraging the Power of MLLMs for Gloss-Free Sign Language Translation
Jungeun Kim (Yonsei University), Ha Young Kim (Yonsei University)
CodeRecognitionGenerationTransformerLarge Language ModelPrompt EngineeringContrastive LearningVideoTextMultimodality
🎯 What it does: A gloss-free sign language translation framework called MMSLT is constructed, utilizing a multimodal large language model (MLLM) to generate sign language descriptions and fuse them with video features, ultimately outputting spoken sentences.
🎯 What it does: A method is proposed to generate animatable 3D human models from a single image within seconds, using 3D Gaussian splatting representation, and directly completing texture and pose recovery in forward inference.
Lifting the Structural Morphing for Wide-Angle Images Rectification: Unified Content and Boundary Modeling
Wenting Luan (National University of Defense Technology), Kang Liao (Nanyang Technological University)
CodeRestorationObject DetectionSegmentationImage
🎯 What it does: A one-stage ConBo-Net is proposed, which jointly corrects the distortion of wide-angle lenses and the image boundaries, achieving both geometric accuracy and rectangular boundaries for distorted images.
🎯 What it does: A lightweight blind super-resolution model LightBSR is proposed, emphasizing the discernibility of implicit degradation representation (IDR);
🎯 What it does: A lightweight real-time image enhancement framework is proposed by decomposing 3D LUT into 2D LUT and utilizing SVD compression, balancing spatial awareness and achieving faster inference.
🎯 What it does: A 3D Gaussian Splatting (3DGS) image upsampling method for lightweight GPUs is proposed, utilizing Gaussian analytical gradients to achieve cubic spline interpolation.
LIRA: Inferring Segmentation in Large Multi-modal Models with Local Interleaved Region Assistance
Zhang Li (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
CodeSegmentationGenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: A new framework called LIRA is proposed, which enhances segmentation accuracy of large models while maintaining text understanding and reducing hallucination generation.
LIRA: Reasoning Reconstruction via Multimodal Large Language Models
Zhen Zhou (Institute of Automation, Chinese Academy of Sciences), Fengshui Jing (Institute of Automation, Chinese Academy of Sciences)
CodeRecognitionObject DetectionSegmentationTransformerLarge Language ModelVision Language ModelMultimodalityPoint CloudBenchmark
🎯 What it does: An online 3D reconstruction task aimed at implicit complex instructions is proposed, along with the LIRA framework to achieve this task.
LLaFEA: Frame-Event Complementary Fusion for Fine-Grained Spatiotemporal Understanding in LMMs
Hanyu Zhou (National University of Singapore), Gim Hee Lee (National University of Singapore)
CodeTransformerLarge Language ModelVision Language ModelVideoMultimodality
🎯 What it does: The LLaFEA framework is proposed, which fuses video information from event cameras and frame cameras to achieve fine-grained spatiotemporal reasoning in LMM.
LLaVA-CoT: Let Vision Language Models Reason Step-by-Step
Guowei Xu (Tsinghua University), Li Yuan (Peking University)
CodeExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityChain-of-Thought
🎯 What it does: LLaVA-CoT is constructed, a visual language model that achieves autonomous structured reasoning through a four-stage process (summary, image description, reasoning, conclusion), and proposes the SWIRES method for re-tracking search during the testing phase.
LLaVA-KD: A Framework of Distilling Multimodal Large Language Models
Yuxuan Cai (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
CodeKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a method to transfer knowledge from large-scale multimodal language models (l-MLLM) to small-scale multimodal language models (s-MLLM) through knowledge distillation, achieving a significant improvement in the performance of lightweight models.
LLaVA-SP: Enhancing Visual Representation with Visual Spatial Tokens for MLLMs
Haoran Lou (Beijing University of Posts and Telecommunications), Xinliang Wang (Beihang University)
CodeRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: Based on the CLIP-ViT visual encoder, six visual spatial tokens are added to the visual tokens to extract multi-scale spatial information through cropping or pooling, and fine-grained features are enhanced by cross-attention fusion, improving the visual understanding capability of the multimodal large language model (MLLM).
LLM-assisted Entropy-based Adaptive Distillation for Unsupervised Fine-grained Visual Representation Learning
Jianfeng Dong (Zhejiang Gongshang University), Xun Wang (Zhejiang Gongshang University)
CodeRetrievalKnowledge DistillationRepresentation LearningConvolutional Neural NetworkLarge Language ModelContrastive LearningImageMultimodality
🎯 What it does: In unsupervised fine-grained visual representation learning, the LEAD framework is proposed, which first uses a large language model to generate fine-grained category descriptions, then utilizes CLIP to generate prototype-driven multimodal knowledge, and adapts the distillation strength through information entropy to achieve a dynamic fusion of teacher model knowledge and self-supervised contrastive learning.
LLM-enhanced Action-aware Multi-modal Prompt Tuning for Image-Text Matching
Mengxiao Tian (Beijing Institute of Technology), Shuo Yang (Shenzhen MSU-BIT University)
CodeRetrievalTransformerLarge Language ModelPrompt EngineeringVision-Language-Action ModelContrastive LearningImageText
🎯 What it does: This paper improves the fine-grained action understanding in image-text matching by allowing CLIP to receive action triplets and action state prompts generated by a large language model, proposing an action-aware multimodal prompt tuning method.
LMM-Det: Make Large Multimodal Models Excel in Object Detection
Jincheng Li (360 AI Research), Yuhui Yin (360 AI Research)
CodeObject DetectionTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: This paper proposes LMM-Det, a method that enables large multimodal models to have end-to-end object detection capabilities without adding dedicated detection modules.
LMM4LMM: Benchmarking and Evaluating Large-multimodal Image Generation with LMMs
Jiarui Wang (Shanghai Jiao Tong University), Xiongkuo Min (Shanghai Jiao Tong University)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningImageMultimodalityBenchmark
🎯 What it does: A large-scale multimodal image generation evaluation database, EvalMi-50K (20 fine-grained tasks, 50,400 generated images, 2 million+ MOS, and 50,000 QA), and a unified evaluation model based on LMM, LMM4LMM, are proposed for simultaneously assessing perceptual quality, text-image correspondence, and task-specific accuracy.
Looking in the Mirror: A Faithful Counterfactual Explanation Method for Interpreting Deep Image Classification Models
Townim Chowdhury, Zhibin Liao (Australian Institute for Machine Learning)
CodeClassificationExplainability and InterpretabilityConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: Proposes the Mirror-CFE method for generating trustworthy and interpretable contrastive explanation images that align with the decision boundaries of depth image classifiers.
🎯 What it does: An AI-generated image detection method called LOTA is proposed, which is based on low-bit plane noise extraction and maximum gradient block selection.
Low-Light Image Enhancement Using Event-Based Illumination Estimation
Lei Sun (Sofia University St. Kliment Ohridski), Luc Van Gool (Sofia University St. Kliment Ohridski)
CodeRestorationImage
🎯 What it does: Utilizing the temporal mapping of events from event cameras to estimate illumination, combined with the Retinex theory to enhance low-light images;
Lumina-Image 2.0: A Unified and Efficient Image Generative Framework
Qi Qin (Shanghai AI Laboratory), Peng Gao (Shenzhen Institutes of Advanced Technology)
CodeGenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageText
🎯 What it does: This paper presents Lumina-Image 2.0, a unified and efficient text-to-image generation framework that integrates the unified Next-DiT model, the unified Captioner UniCap, and multi-stage progressive training and inference acceleration strategies.
🎯 What it does: The paper proposes a fast infrared-visible image fusion framework called LUT-Fuse based on a learnable lookup table (MM-LUT), utilizing low-order approximation coding and high-order context coding, and achieving efficient fusion through distillation learning.
LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agents
Boyu Chen (Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences), Yali Wang (Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences)
CodeRetrievalTransformerLarge Language ModelAgentic AIVision Language ModelVideoMultimodality
🎯 What it does: Proposes LVAgent, a multi-round dynamic collaboration multimodal large language model agent framework for long video understanding.
LVFace: Progressive Cluster Optimization for Large Vision Models in Face Recognition
Jinghan You (ByteDance Inc), Jiao Ran (ByteDance Inc)
CodeRecognitionOptimizationTransformerImage
🎯 What it does: This paper proposes LVFace, a face recognition model based on Vision Transformer, and introduces a Progressive Cluster Optimization (PCO) training framework.
MA-CIR: A Multimodal Arithmetic Benchmark for Composed Image Retrieval
Jaeseok Byun (Seoul National University), Taesup Moon (Seoul National University)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: The MA-CIR benchmark is proposed to evaluate the combinatorial understanding ability of synthetic image retrieval models in scenarios involving three arithmetic types (addition, elimination, replacement) and seven complex semantic categories (left-right, up-down, spatial reasoning, color, size, action, object reasoning). High-quality retrieval triplets are constructed through manual review and hard negative samples; at the same time, improvements are made to the text encoder based on large language models (LLM), which are fine-tuned on MA-CIR.
🎯 What it does: Generate geometrically consistent high-quality 3D city models from satellite images by first extracting geometric and texture features, then using a dual-encoder to encode and generate multi-view images through a city-scale multi-view diffusion model, and finally reconstructing the final 3D city using 3D Gaussian splatting.
CodePose EstimationRecurrent Neural NetworkTime Series
🎯 What it does: This paper proposes an IMU attitude estimation method named MagShield, specifically designed to address attitude errors caused by magnetic interference in sparse inertial motion capture systems.
Mamba-3VL: Taming State Space Model for 3D Vision Language Learning
Yuan Wang (Tsinghua University), Zhipeng Zhang (Anyverse Intelligence)
CodeRecognitionObject DetectionSegmentationRobotic IntelligenceTransformerPrompt EngineeringVision Language ModelMultimodalityPoint Cloud
🎯 What it does: Mamba-3VL is proposed, a 3D vision-language framework based on the state space model (Mamba), designed to unify the handling of interactive tasks between 3D vision and natural language (such as visual localization, dense description, question answering, segmentation, etc.)
MamV2XCalib: V2X-based Target-less Infrastructure Camera Calibration with State Space Model
Yaoye Zhu (Institute for AI Industry Research), Yan Wang (Institute for AI Industry Research)
CodeAutonomous DrivingConvolutional Neural NetworkRecurrent Neural NetworkSimultaneous Localization and MappingPoint Cloud
🎯 What it does: A targetless baseline camera calibration method based on vehicle-side LiDAR, MamV2XCalib, is proposed, which utilizes vehicle motion data to achieve automatic rotational calibration of infrastructure cameras.
🎯 What it does: This paper presents MaskSAM, a prompt-free, mask classification-based Segment Anything Model (SAM) adaptation framework for voxel segmentation in medical imaging; it achieves automatic prompting and predicts semantic labels by generating auxiliary masks, bounding boxes, and classification tags.
MaTVLM: Hybrid Mamba-Transformer for Efficient Vision-Language Modeling
Yingyue Li (Huazhong University of Science and Technology), Xinggang Wang (Huazhong University of Science and Technology)
CodeComputational EfficiencyKnowledge DistillationTransformerVision Language ModelMultimodality
🎯 What it does: Knowledge distillation is performed on a pre-trained vision-language model to construct the MaTVLM hybrid Mamba-Transformer architecture.
🎯 What it does: This paper proposes MAVFlow, which utilizes dual-modal guidance (audio speaker embedding + visual emotion embedding) and conditional flow matching (OT-CFM) to achieve zero-shot multilingual audio-visual translation while maintaining speaker identity and emotional consistency.
🎯 What it does: A novel detection framework called MessDet is proposed, which strictly maintains rotational equivariance. By improving downsampling, introducing rotational equivariant channel attention, and utilizing a multi-branch head network, it achieves more efficient and accurate object detection in aerial images.
Yijun Yang (Hong Kong University of Science and Technology), Jieneng Chen
CodeSegmentationGenerationOptimizationConvolutional Neural NetworkTransformerLarge Language ModelVision Language ModelDiffusion modelContrastive LearningWorld ModelImageBiomedical DataComputed Tomography
🎯 What it does: Proposes the Medical World Model (MeWM), which predicts post-processed tumor images from pre-processed CT scans and assists in TACE treatment decision-making through a visual language strategy model, tumor generation dynamics model, and inverse dynamics survival analysis model.
MedVSR: Medical Video Super-Resolution with Cross State-Space Propagation
Xinyu Liu (Chinese University of Hong Kong), Ender Konukoglu (ETH Zurich)
CodeRestorationSuper ResolutionOptical FlowVideoBiomedical Data
🎯 What it does: Proposes the MedVSR framework, which addresses the characteristics of medical videos such as frame skipping and jitter, using Cross-State Space Propagation (CSSP) and Internal State Space Reconstruction (ISSR) to achieve high-quality video super-resolution.
🎯 What it does: The MEGA framework is proposed to achieve memory-efficient 4D Gaussian Splatting for dynamic scene reconstruction and real-time rendering.
METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Models
Yuchen Liu (Shanghai Jiao Tong University), Qi Tian (Huawei Inc.)
CodeRecognitionComputational EfficiencyTransformerVision Language ModelImageTextMultimodality
🎯 What it does: The METEOR framework for multi-encoder collaborative token pruning is proposed, which can gradually remove redundant visual tokens in the three stages of visual encoding, fusion, and LLM decoding.
🎯 What it does: This paper proposes Metric Convolution based on image manifolds, which achieves adaptive convolution kernels by sampling unit spheres (tangent spaces or geodesics) at each pixel, unifying and extending existing variable convolution methods.
MGSfM: Multi-Camera Geometry Driven Global Structure-from-Motion
Peilin Tao (Institute of Automation, Chinese Academy of Sciences), Shuhan Shen (Institute of Automation, Chinese Academy of Sciences)
CodePose EstimationAutonomous DrivingOptimizationSimultaneous Localization and MappingImageVideo
🎯 What it does: A multi-camera driven global structure from motion (SfM) framework called MGSfM is proposed, which combines decoupled rotation averaging and mixed translation averaging to achieve efficient and robust global camera pose estimation and 3D reconstruction.
🎯 What it does: A dual-branch mutually reinforcing 2D/3D Gaussian scattering framework MGSR is proposed for achieving high-fidelity surface reconstruction and novel view synthesis under varying lighting conditions.
🎯 What it does: A multi-hypothesis temporal prediction scheme (MH-LVC) is proposed in learning-based video compression, which constructs a temporal predictor by simultaneously using short-term recent frames and long-term key frames, and utilizes the prediction results for conditional residual coding.
🎯 What it does: This paper proposes a new image-point cloud registration framework MinCD-PnP and MinCD-Net, which utilizes the minimization of the Chamfer distance approximating blind PnP to learn the 2D-3D correspondence, thereby achieving high-precision camera pose estimation.
CodeTransformerLarge Language ModelPrompt EngineeringVideoMultimodality
🎯 What it does: The MINERVA dataset was constructed, containing 1515 questions from multiple domains and varying video lengths, each with 5 answer options and manually crafted multi-step reasoning trajectories to evaluate the complex video reasoning capabilities of multimodal models.
MIORe & VAR-MIORe: Benchmarks to Push the Boundaries of Restoration
George Ciubotariu (University of Wurzburg), Radu Timofte (University of Wurzburg)
CodeRestorationOptical FlowVideoBenchmark
🎯 What it does: Two multi-task datasets, MIORe and VAR-MIORe, are proposed, covering motion recovery tasks from low to extreme motion amplitudes, and algorithms such as deblurring, frame interpolation, and optical flow are evaluated on them.
🎯 What it does: This paper reveals the phenomenon of Catastrophic Overfitting (CO) in Fast Adversarial Training by studying the transferability of label information across samples in single-step adversarial perturbations, and proposes the Label Information Elimination Training (LIET) method to actively eliminate such label information, thereby reducing the risk of CO.
Mitigating Object Hallucinations via Sentence-Level Early Intervention
Shangpin Peng (Harbin Institute of Technology), Zhuotao Tian (Harbin Institute of Technology)
CodeObject DetectionLarge Language ModelImageTextMultimodality
🎯 What it does: This paper proposes the SENTINEL framework, which addresses the object hallucination problem in multimodal large language models by employing sentence-level early intervention to suppress the spread of hallucinations.
Mixture of Experts Guided by Gaussian Splatters Matters: A new Approach to Weakly-Supervised Video Anomaly Detection
Giacomo D' Amicantonio, Egor Bondarev (Eindhoven University of Technology)
CodeAnomaly DetectionTransformerMixture of ExpertsGaussian SplattingVideoMultimodality
🎯 What it does: A multi-expert (Mixture-of-Experts) framework based on Gaussian splatting is proposed, utilizing Temporal Gaussian Splatting loss to enhance the temporal integrity of weakly supervised video anomaly detection, and integrating experts with coarse-grained features through a gating model.
Mixture-of-Scores: Robust Image-Text Data Valuation via Three Lines of Code
Sitong Wu (Chinese University of Hong Kong), Jiaya Jia (Hong Kong University of Science and Technology)
CodeData-Centric LearningTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper studies the differences between image-text pairing quality scoring models and proposes the Mixture-of-Scores (MoS) method, which integrates multiple scoring models to achieve a more robust quality assessment.
MM-IFEngine: Towards Multimodal Instruction Following
Shengyuan Ding (Fudan University), Jiaqi Wang (Fudan University)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningImageTextMultimodalityBenchmark
🎯 What it does: Proposed the MM-IFEngine pipeline, which automatically generates image-instruction pairs with various constraints, and based on this, constructs MM-IFInstruct-23k (SFT data), MM-IFDPO-23k (DPO data), and the MM-IFEval evaluation benchmark;
MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs
Erik Daxberger (Apple), Peter Grasch (Apple)
CodeRecognitionGenerationDepth EstimationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodalityPoint Cloud
🎯 What it does: This paper proposes a multimodal large language model, MM-Spatial, for understanding three-dimensional space, and creates a new VQA dataset, CA-VQA, through a generation pipeline based on high-quality 3D scene data. The model is then fine-tuned with supervision on this dataset and evaluated on multiple 3D spatial tasks.
MMAD: Multi-label Micro-Action Detection in Videos
Kun Li (Hefei University of Technology), Meng Wang (Zhejiang University)
CodeRecognitionObject DetectionTransformerVideo
🎯 What it does: This paper proposes the task of Multi-Label Micro Action Detection (MMAD) and constructs the MMA-52 dataset, providing a baseline model and conducting experimental evaluations.
MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
Tianhong Gao (Baidu Inc), Gang Zhang (Baidu Inc)
CodeOptimizationData-Centric LearningTransformerLarge Language ModelTextMultimodalityRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: We constructed and released MMAT-1M, the first multimodal agent tuning dataset at a million scale, which includes reasoning steps, tool calls, and reflection processes generated based on GPT-4o, and provides both one-round (ORR) and multi-round (RR) formats; at the same time, we evaluated the model's performance improvement on eight major public benchmarks and Dyn-VQA.
MMCR: Benchmarking Cross-Source Reasoning in Scientific Papers
Yang Tian (Shanghai Jiao Tong University), Bo Zhao (Shanghai Jiao Tong University)
CodeVision Language ModelTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: A benchmark called MMCR was constructed, with 276 manually annotated cross-source reasoning questions to evaluate the cross-source reasoning ability of VLM in scientific papers.
🎯 What it does: This paper proposes MMGEO, which utilizes multi-modal combined queries (image + point cloud/depth/text) to achieve UAV geographic positioning in satellite image retrieval databases.
🎯 What it does: An image-to-video diffusion model called MobileVD has been implemented on mobile devices, capable of quickly generating 14 frames of 512×256 resolution video in low-power environments.
🎯 What it does: For the video diffusion model DiT, we propose the ProfilingDiT mechanism, which achieves adaptive feature caching based on attention and noise variations during forward inference, significantly reducing inference costs without sacrificing visual quality.
🎯 What it does: This paper proposes the Moment Quantization (MQVTG) method, which maps continuous video features to discrete codewords through vector quantization, enhancing the discriminative ability between relevant and irrelevant segments in video temporal localization.
Jianfei Jiang (University of Science and Technology Beijing), Huimin Ma (University of Science and Technology Beijing)
CodeDepth EstimationTransformerPoint Cloud
🎯 What it does: This paper proposes MonoMVSNet, a multi-view stereo network guided by monocular priors, which enhances multi-view matching and depth estimation using features and depth information from a monocular depth model.
🎯 What it does: Proposes an automatic annotation and training framework for training 3D object detection models from a monocular camera without manual labeling and without LiDAR.
🎯 What it does: A two-stage diffusion-based controllable motion generation framework is proposed, which first generates low-dimensional key joint trajectories using sparse key joint control signals, and then completes natural full-body motion through the complete key joint trajectories, while supporting time-independent control.
MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agent
Xinyao Liao (Nanyang Technological University), Chi Zhang (Westlake University)
CodeObject DetectionGenerationDepth EstimationTransformerLarge Language ModelDiffusion modelSimultaneous Localization and MappingOptical FlowImageVideoTextBenchmark
🎯 What it does: This paper proposes MotionAgent, which utilizes LLM and object detection to parse motion information from text, generating object trajectories and camera extrinsics, and then achieves text-driven fine-grained image-to-video generation through optical flow synthesis and an optical flow adapter.
🎯 What it does: MotionDiff proposes a training-free zero-shot copy flow-assisted diffusion framework that can achieve complex motion editing such as translation, scaling, rotation, and stretching in multi-view static scenes.
🎯 What it does: This paper proposes MotionFollower, a video motion editing framework based on diffusion models, which can transfer the motion of a target video to a source video while keeping the background, character appearance, and camera motion of the source video unchanged.
🎯 What it does: A multi-prompt (spectral, text, visual) framework MP-HSIR is proposed for integrated hyperspectral image denoising, deblurring, super-resolution, inpainting, dehazing, band completion, and other nine tasks.
MPG-SAM 2: Adapting SAM 2 with Mask Priors and Global Context for Referring Video Object Segmentation
Fu Rong (Wuhan University), Lefei Zhang (Horizon Robotics)
CodeObject DetectionSegmentationTransformerVision Language ModelVideoMultimodality
🎯 What it does: A new RVOS framework called MPG-SAM 2 based on SAM 2 is proposed, which utilizes a multimodal encoder to generate video-text aligned features and enhances segmentation through mask priors to generate dense prompts and hierarchical global historical aggregation.
🎯 What it does: This study constructed a controllable generated synthetic facial image dataset called SynFIQA, and utilized multi-domain reference representations (recognition embeddings, spatial, visual-language) to train and annotate a facial image quality assessment (FIQA) model.
🎯 What it does: This paper proposes a pseudo-label enhanced audio-visual Mamba network (MUG), which improves the accuracy of audio and visual video parsing through cross-modal random combination data augmentation and text semantic interaction.
🎯 What it does: A multi-baseline generalizable 3D Gaussian splatting method, MuGS, is proposed for novel view synthesis, capable of generating high-quality images uniformly under sparse input and different baselines (small and large baselines).
CodeCompressionComputational EfficiencyTransformerLarge Language ModelVision Language ModelVideo
🎯 What it does: A training-independent multi-granularity spatiotemporal token merging method (STTM) is proposed in video large language models to reduce the number of visual tokens and improve inference speed.
Zijia Lu (Northeastern University), Ehsan Elhamifar (Northeastern University)
CodeRecognitionSegmentationGraph Neural NetworkTransformerVision Language ModelVideoTextMultimodality
🎯 What it does: This paper proposes the Multi-modal Few-shot Temporal Action Segmentation (MMF-TAS) task and achieves adaptive segmentation for the new task through a Prototype Graph Network.
Multi-modal Segment Anything Model for Camouflaged Scene Segmentation
Guangyu Ren (Xi'an Jiaotong-Liverpool University), Tania Stathaki (Imperial College London)
CodeObject DetectionSegmentationTransformerVision Language ModelImageMultimodalityBenchmark
🎯 What it does: For target segmentation in concealed scene images, a multi-modal prompt Segment Anything model (MM-SAM) is proposed, which completes segmentation without the need for manual boxes or point prompts.
Multi-View Slot Attention Using Paraphrased Texts for Face Anti-Spoofing
Jeongmin Yu (Yonsei University), Ha Young Kim (Yonsei University)
CodeRecognitionDomain AdaptationRecurrent Neural NetworkPrompt EngineeringVision Language ModelImageVideo
🎯 What it does: The MVP-FAS framework is proposed, achieving cross-domain generalization for face spoofing detection through multi-view slot attention and multi-text patch alignment.
🎯 What it does: This paper proposes Multi-Dimensional Byte Pair Encoding (MDBPE), which compresses the sequence length of visual data by statistically merging frequent token pairs in two-dimensional (or three-dimensional) space, allowing Transformers to train and infer faster and more efficiently.