IEEE/CVF Conference on Computer Vision and Pattern Recognition · 851 papers
UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editing
Yiheng Li (Chinese Academy of Sciences), Xilin Chen (Chinese Academy of Sciences)
CodeGenerationPose EstimationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: The UniPose framework is proposed, achieving simultaneous understanding, generation, and editing of human poses within a unified multimodal model.
🎯 What it does: A unified 3D point cloud pre-training framework called UniPre3D is proposed, which is applicable to both object-level and scene-level point clouds.
🎯 What it does: This paper proposes UniVAD, a training-agnostic, unified few-shot visual anomaly detection method that can detect anomalies in industrial, logical, and medical fields without the need to train models for each domain, requiring only a small number of normal samples during testing.
Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter
Zhengyi Zhong (National University of Defense Technology), Wei Yang Bryan Lim (Nanyang Technological University)
CodeFederated LearningSafty and PrivacyComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: A reversible federated no-learning framework FUSED is proposed, which constructs sparse adapters at key layers to cover forgotten knowledge without altering the original model;
🎯 What it does: A multi-modal media manipulation detection and localization framework based on consistency learning, named CSCL, is proposed, specifically targeting fine-grained forgery detection and localization tasks for images and text.
🎯 What it does: This paper proposes the UGP framework, which removes cross-attention and incorporates progressive self-attention and BEV features to enhance the generalization ability of LiDAR point cloud registration in cross-distance and cross-dataset scenarios.
Unlocking the Potential of Unlabeled Data in Semi-Supervised Domain Generalization
Dongkwan Lee (Seoul National University), Nojun Kwak (Seoul National University)
CodeDomain AdaptationContrastive LearningImage
🎯 What it does: This paper proposes a module named UPCSC, which utilizes unlabeled samples with low model confidence in semi-supervised domain generalization to enhance the model's performance in unseen domains.
🎯 What it does: The UNA model is proposed, utilizing fluid-driven anomaly randomization to achieve healthy reconstruction and anomaly detection of multimodal (CT/MRI) brain structures.
Unsupervised Discovery of Facial Landmarks and Head Pose
Satyajit Tourani (International Institute of Information Technology Hyderabad), Muhammad Haris Khan (Mohamed bin Zayed University of Artificial Intelligence)
🎯 What it does: This paper proposes an unsupervised method utilizing Stable Diffusion to jointly discover facial key points and head poses, refining key point localization through self-supervised cyclic consistency, image-aware text embedding, and 3D-based rotation rendering enhancement.
Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning
Tim Lenz (EKFZ for Digital Health TU Dresden), Jakob N. Kather (EKFZ for Digital Health TU Dresden)
CodeRepresentation LearningTransformerContrastive LearningImageBiomedical Data
🎯 What it does: A novel unsupervised contrastive learning framework called COBRA is proposed, which generates full-slice low-dimensional representations in the feature space using multi-scale features from multiple base models.
🎯 What it does: This paper proposes a scalable differential clustering method based on random Fourier features, called FINC, to identify sample types with significant differences in generation frequency between two generative models.
Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly
Yexin Liu (Hong Kong University of Science and Technology), Bo Zhao (Shanghai Jiao Tong University)
CodeTransformerLarge Language ModelPrompt EngineeringMultimodalityBenchmark
🎯 What it does: This paper proposes the MMVU benchmark to evaluate the phenomenon where multimodal large language models still provide incorrect answers after visual understanding, and enhances the model's robustness by constructing positive and negative paired samples and prompt strategies.
🎯 What it does: This paper proposes a multi-state unified low-light image enhancement and deblurring model URWKV, designed to handle dynamically coupled low-light and blur degradation.
Using Powerful Prior Knowledge of Diffusion Model in Deep Unfolding Networks for Image Compressive Sensing
Chen Liao (Beijing Jiaotong University), Zhongli Wang (Beijing Jiaotong University)
CodeRestorationCompressionDiffusion modelImage
🎯 What it does: A deep unfolding network DMP-DUN based on a pre-trained diffusion model is proposed, achieving high-quality reconstruction of image compressed sensing with very few iterations (only 2 steps).
CodeRestorationDepth EstimationConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: An unsupervised perspective mutual mapping framework V2V3D is proposed for image denoising and 3D reconstruction in light field microscopy.
🎯 What it does: Designed and released the first V2X collaborative perception dataset containing 4D radar, V2X-R, and implemented a LiDAR-4D radar multimodal fusion framework based on this dataset, while also proposing a Multi-modal Denoising Diffusion (MDD) denoising module.
VELOCITI: Benchmarking Video-Language Compositional Reasoning with Strict Entailment
Darshana Saravanan (International Institute of Information Technology Hyderabad), Makarand Tapaswi (International Institute of Information Technology Hyderabad)
CodeTransformerLarge Language ModelVision Language ModelVideoTextBenchmark
🎯 What it does: Designed and released the VELOCITI benchmark to evaluate the reasoning ability of video-language models in short videos regarding agents, actions, and multi-event combinations.
🎯 What it does: This paper proposes vesselFM, a foundational model for 3D vascular segmentation based on three heterogeneous data sources (real vascular images, domain-randomized synthetic images, and flow-matching generated images);
🎯 What it does: This paper presents the Visual Geometry Grounded Transformer (VGGT), a large feedforward transformer capable of predicting camera parameters, depth maps, point clouds, and point tracking, among other 3D attributes, from any number of views at once.
Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding
Duo Zheng (Chinese University of Hong Kong), Liwei Wang (Chinese University of Hong Kong)
CodeRecognitionRepresentation LearningTransformerLarge Language ModelContrastive LearningVideoPoint Cloud
🎯 What it does: This paper proposes Video-3D LLM, which directly constructs location-aware video representations using video frames and corresponding 3D coordinates to accomplish tasks such as 3D visual localization, dense description, and question answering.
Video-Bench: Human-Aligned Video Generation Benchmark
Hui Han (Shanghai Jiao Tong University), Yongxin Ni (National University of Singapore)
CodeGenerationData SynthesisTransformerLarge Language ModelVideoTextBenchmarkChain-of-Thought
🎯 What it does: This paper presents Video-Bench, a comprehensive evaluation benchmark for video generation that combines a multi-dimensional evaluation system (video-condition alignment and video quality) and achieves automated assessment through a multi-modal large language model (MLLM);
Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding
Yan Shu (Shanghai Jiao Tong University), Bo Zhao (Shanghai Jiao Tong University)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: This study focuses on long video understanding and proposes the Video-XL model, which utilizes Visual Summary Tokens (VST) to compress videos to fit the context length limitations of large multimodal language models.
VideoGEM: Training-free Action Grounding in Videos
Felix Vogel (Goethe University Frankfurt), Hilde Kuehne (MPI for Informatics)
CodeRecognitionObject DetectionTransformerPrompt EngineeringVision Language ModelVideoText
🎯 What it does: A training-free visual language model video action localization method called VideoGEM is proposed, which can directly utilize pre-trained CLIP, OpenCLIP, and ViCLIP to spatially locate actions in videos.
VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understanding
Kangsan Kim (Korea Advanced Institute of Science and Technology), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
CodeClassificationRecognitionDomain AdaptationTransformerVision Language ModelVideoTextMultimodality
🎯 What it does: A training-agnostic video context learning framework (VideoICL) for out-of-distribution (OOD) video tasks is proposed, which enhances the generalization ability of video multimodal models by dynamically selecting the most relevant examples during inference and performing iterative reasoning.
Vision-Language Model IP Protection via Prompt-based Learning
Lianyu Wang (Key Laboratory of Brain-Machine Intelligence Technology Ministry of Education China), Daoqiang Zhang (Key Laboratory of Brain-Machine Intelligence Technology Ministry of Education China)
CodeClassificationDomain AdaptationSafty and PrivacyTransformerPrompt EngineeringVision Language ModelContrastive LearningImage
🎯 What it does: This paper proposes a prompt-based IP protection framework called IP-CLIP, designed to limit the reasoning capabilities of visual language models like CLIP in unauthorized domains.
VisionArena: 230k Real World User-VLM Conversations with Preference Labels
Christopher Chou (Stanford University), Wei-Lin Chiang (University of California Berkeley)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelTextMultimodalityBenchmark
🎯 What it does: This paper constructs the VisionArena dataset, collecting 230K dialogues between real users and visual language models (VLMs), and generates a VLM competition ranking through user preference voting.
🎯 What it does: VISTA3D is proposed, a unified 3D medical image segmentation foundation model that achieves high-precision automatic segmentation across 127 supported categories and enables interactive error correction and zero-shot segmentation through 3D point annotations.
🎯 What it does: The VISTREAM framework is researched and proposed, achieving efficient inference of visual flow perception through differential coding and LoCC-SNN.
🎯 What it does: This paper proposes a co-salient object detection method based on Visual Consensus Prompt (VCP), freezing the pre-trained Transformer and only adjusting a minimal number of prompt parameters to complete the CoSOD task.
ViUniT: Visual Unit Tests for More Robust Visual Programming
Artemis Panagopoulou (Salesforce AI Research), Juan Carlos Niebles (University of Pennsylvania)
CodeGenerationData SynthesisOptimizationTransformerLarge Language ModelReinforcement LearningDiffusion modelImageTextMultimodality
🎯 What it does: The VuniT framework is proposed to enhance the reliability and correctness of visual programs by automatically generating visual unit tests.
VL2Lite: Task-Specific Knowledge Distillation from Large Vision-Language Models to Lightweight Networks
Jinseong Jang (SK Telecom), Byeongwon Lee (SK Telecom)
CodeClassificationKnowledge DistillationConvolutional Neural NetworkVision Language ModelImageMultimodality
🎯 What it does: A single-stage knowledge distillation framework named VL2Lite is proposed, which directly transfers the knowledge of pre-trained vision-language models to lightweight networks to enhance classification performance.
🎯 What it does: A Transformer structure named VolFormer is proposed for the super-resolution and denoising recovery tasks of single high-resolution hyperspectral images.
Volumetric Surfaces: Representing Fuzzy Geometries with Layered Meshes
Stefano Esposito (University of Tuebingen), Andreas Geiger (University of Tuebingen)
CodeGenerationComputational EfficiencyMesh
🎯 What it does: A multi-layer SDF layered representation (k-SDF) is designed to achieve real-time rendering with a small number of sampling points by transforming fuzzy geometry into several translucent mesh layers.
🎯 What it does: This work improves the alpha computation method of the 3D Gaussian Splatting (3DGS) efficient rasterization framework while maintaining its efficiency, by performing analytical integration of Gaussian density directly in 3D space, resulting in more physically accurate transmittance that can be directly used for color synthesis.
🎯 What it does: For the word-level sign language recognition task, VSNet is proposed, which first obtains a simplified skeleton through adaptive skeleton simplification (weak joint dropping), then groups the skeleton into visual symbols (VS), and utilizes a self-attention model to capture the spatial-temporal relationships of VS, ultimately achieving sign language recognition.
🎯 What it does: A plugin semantic decoupling module named Wav2Sem is proposed, which extracts semantic features from speech using global semantic information and integrates them with a self-supervised audio encoder to improve the coupling and averaging issues of near-homophones in 3D speech-driven facial animation.
🎯 What it does: The WAVE method is proposed, utilizing shared weight templates and differentiable size-specific scalers to initialize visual Transformer models of different scales from a multi-task perspective.
🎯 What it does: This paper proposes Weakly Supervised Contrastive Adversarial Training (WSCAT), which generates complete perturbation adversarial samples on semi-supervised data to block the correlation between non-robust features and labels, thereby learning more robust features; it also provides theoretical analysis and experimental validation.
Weakly Supervised Semantic Segmentation via Progressive Confidence Region Expansion
Xiangfeng Xu (East China Normal University), Shaohui Lin (East China Normal University)
CodeSegmentationTransformerImage
🎯 What it does: A weakly supervised semantic segmentation framework based on Progressive Confidence Region Expansion (PCRE) is proposed to address the issue of over-expansion in CAM generated by ViT.
🎯 What it does: Proposes the WeakMCN multi-task collaborative network, achieving joint learning of weakly supervised reference expression comprehension (WREC) and segmentation (WRES).
🎯 What it does: A WF-VAE based on multi-layer wavelet transform is designed, utilizing low-frequency energy flow to compress video information into latent space, and a Causal Cache is proposed to achieve lossless block-level inference.
🎯 What it does: The study investigates which alternative data can effectively transfer teacher knowledge during knowledge distillation when the original training data is unavailable, and proposes criteria for measuring the quality of distillation data.
🎯 What it does: A self-supervised video representation learning framework T-CoRe is proposed, which guides the recovery of masked video models through temporal correspondence, significantly improving the performance of video downstream tasks.
🎯 What it does: A framework for instance segmentation called WISH is proposed, which can simultaneously utilize multiple types of weak labels and supports heterogeneous weak annotations.
🎯 What it does: A dataset called ShanghaiWaste was constructed, containing 12,208 waste images and 40,392 patch-level annotations, and a weakly supervised pseudo-label generation framework WISNet was proposed for achieving pixel-level semantic segmentation of waste images.
Words or Vision: Do Vision-Language Models Have Blind Faith in Text?
Ailin Deng (National University of Singapore), Bryan Hooi (National University of Singapore)
CodeTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This study investigates the modal preference of visual-language models (VLM) when there is inconsistency between visual and textual information. It constructs a benchmark with four types of tasks, including matching, corruption, and irrelevant text, to evaluate 10 VLMs. The findings reveal a phenomenon of 'blind trust in text' and explore the impacts of factors such as prompts, model size, text relevance, token order, and unimodal confidence. Subsequently, it proposes to mitigate this bias through supervised fine-tuning combined with text augmentation.
Your Scale Factors are My Weapon: Targeted Bit-Flip Attacks on Vision Transformers via Scale Factor Manipulation
Jialai Wang (National University of Singapore), Zhenkai Liang (Tsinghua University)
CodeAdversarial AttackTransformerImage
🎯 What it does: This paper proposes a target bit-flip attack method Flip-S aimed at quantized Transformers (ViT), which utilizes the flippable bits of the scale factor in quantized models to implant backdoors and induce the model to output a specified category for target inputs.
🎯 What it does: This paper proposes a zero-shot image restoration framework based on a Consistency Model (CM4IR), capable of completing super-resolution, deblurring, and inpainting tasks with only 4 neural function evaluations (NFE).