IEEE/CVF International Conference on Computer Vision Β· 833 papers
ViT-Split: Unleashing the Power of Vision Foundation Models via Efficient Splitting Heads
Yifan Li (Michigan State University), Liu Ren (Bosch Research North America & Bosch Center for Artificial Intelligence)
CodeObject DetectionSegmentationTransformerImage
π― What it does: This paper proposes a visual foundation model adapter named ViT-Split, aimed at enhancing multi-task downstream performance solely through two lightweight heads (Prior head and Task head) while keeping the original VFM parameters frozen.
VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Models
Jiacheng Ruan (Shanghai Jiao Tong University), Yuzhuo Fu
CodeTransformerLarge Language ModelVision Language ModelMultimodalityBenchmark
π― What it does: This paper presents VLRMBench, a visual-language reward model evaluation benchmark that includes 12 tasks and 12,634 questions, aimed at comprehensively examining the model's process understanding, result judgment, and critical generation capabilities.
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVideoBenchmark
π― What it does: Developed the VMBench video motion assessment benchmark, which includes five-dimensional human perception-aligned motion quality metrics and a large-scale diverse motion prompt library.
CodeObject DetectionObject TrackingKnowledge DistillationPrompt EngineeringVision Language ModelContrastive LearningVideo
π― What it does: This paper presents VOVTrack, an end-to-end framework for open vocabulary multi-object tracking (OVMOT) that integrates three sub-tasks: object detection, classification, and association.
CodeGenerationOptimizationSafty and PrivacyLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringVideoText
π― What it does: This paper proposes the VPO framework, addressing the prompt optimization issue for text-to-video generation models. It constructs an alignment mechanism based on the three principles of 'harmless, accurate, and useful' and achieves safe, accurate, and helpful prompt generation through a two-stage process (principle-driven SFT and multi-feedback preference optimization).
VPR-Cloak: A First Look at Privacy Cloak Against Visual Place Recognition
Shuting Dong (Tsinghua University), Chun Yuan (Tsinghua University)
CodeRecognitionOptimizationSafty and PrivacyTransformerContrastive LearningImage
π― What it does: In the visual place recognition (VPR) system, a privacy protection mechanism called VPR-Cloak is proposed, which generates imperceptible perturbations to suppress the identification of the opponent's model location information.
π― What it does: A knowledge distillation framework through Virtual Relation Matching (VRM) is proposed, which constructs a dense relation graph using generated virtual views, and subsequently transfers the student model through a pruning method.
π― What it does: This paper researches and implements a non-causal state space model for visual tasks, VSSD, improving the original SSD to NC-SSD and constructing an efficient visual network.
Vulnerability-Aware Spatio-Temporal Learning for Generalizable Deepfake Video Detection
Dat Nguyen (University of Luxembourg), Djamila Aouada (University of Luxembourg)
CodeClassificationData SynthesisTransformerVideo
π― What it does: The FakeSTormer framework is proposed, which captures spatial and temporal vulnerabilities in a fine-grained manner through multi-task learning, enhancing the generalization ability of deepfake video detection.
What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learning
Chi-Hsi Kung (Indiana University), Yi-Hsuan Tsai (Atmanity Inc.)
CodeRetrievalRepresentation LearningLarge Language ModelContrastive LearningVideoText
π― What it does: By using the action state descriptions generated by large language models and corresponding counterfactuals (hypothetical incorrect or missing operation results) as supervision signals, a hierarchical contrastive learning framework is constructed, enabling the video encoder to capture the real changes and potential variations of actions in the scene, thus achieving a deeper understanding of programmatic videos.
What Makes for Text to 360-degree Panorama Generation with Stable Diffusion?
Jinhong Ni (Australian National University), Jing Zhang (Australian National University)
CodeGenerationData SynthesisTransformerMixture of ExpertsDiffusion modelImage
π― What it does: This paper focuses on the generation of 360Β° panoramic images from text, fine-tuning the Stable Diffusion model with LoRA, and proposes a single-branch framework called UniPano that only adjusts the value and output matrices in the attention module, providing a mechanism explanation for panoramic structure learning.
π― What it does: This study proposes an unsupervised 3D gaze estimation method that generates realistic facial images and accurate pseudo-gaze labels from facial images using controllable 3D Gaussian rendering, thereby training a gaze estimator.
π― What it does: This paper proposes CDiffLane, a lane detection framework based on cold diffusion, which gradually refines lane predictions through a multi-step iterative approach starting from preset anchor points, and introduces resolution scheduling and a learnable Ξ± parameter to achieve fine-tuning from coarse to fine.
π― What it does: A new pseudo-label selection method (Confidence Separable Learning, CSL) is proposed, which dynamically separates reliable and unreliable pixels by constructing a confidence distribution feature space and performing convex optimization; it also introduces Trusted Mask Perturbation to enhance contextual learning in low-confidence areas.
When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning
Junwei Luo (Wuhan University), Yansheng Li (Wuhan University)
CodeRecognitionComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelVision Language ModelImageMultimodalityBenchmark
π― What it does: This paper proposes a text-guided token pruning method that combines dynamic image pyramids for efficient understanding of large-scale remote sensing images.
When Pixel Difference Patterns Meet ViT: PiDiViT for Few-Shot Object Detection
Hongliang Zhou (National University of Defense Technology), Li Liu (National University of Defense Technology)
CodeObject DetectionTransformerImage
π― What it does: Proposes the PiDiViT model, which incorporates a pixel difference convolution fusion module (DCFM) and a multi-scale feature fusion module (MFFM) for few-shot object detection, improving the pre-trained features of ViT.
Where am I? Cross-View Geo-localization with Natural Language Descriptions
Junyan Ye (Sun Yat-Sen University), Weijia Li (Sun Yat-Sen University)
CodeRetrievalExplainability and InterpretabilityTransformerLarge Language ModelContrastive LearningImageTextMultimodality
π― What it does: A cross-perspective geographic localization task based on natural language descriptions is proposed, and a method for locating text-retrieved satellite images/OSM images is implemented.
CodeAutonomous DrivingExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningImageVideoTextMultimodalityChain-of-Thought
π― What it does: An explainable driving attention prediction framework called LLada is proposed, which jointly predicts the focus area (Where), focus semantics (What), and attention reasons (Why).
π― What it does: This paper constructs the largest AI-generated talking head (AGTH) quality assessment dataset THQAβ10K and proposes a novel objective evaluation method FSCD based on the first frame, Y-T slices, and audio-visual consistency.
Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context
Ge Zheng (Sun Yat-sen University), Sibei Yang (ShanghaiTech University)
CodeGenerationTransformerVision Language ModelContrastive LearningTextMultimodality
π― What it does: This study investigates the hallucination problem that arises in large visual language models during long text generation. It proposes an active hallucination instance induction-detection-suppression framework (HalTrapper) that induces hallucination instances, detects them based on two main hypotheses: attention similarity and image context integrity, and suppresses hallucinations during the decoding phase using contrastive context decoding.
π― What it does: A pipeline called Wide2Long is proposed to convert wide-angle images into telephoto images, simulating lens compression and perspective adjustment.
π― What it does: Using wildlife observation data along with multimodal information from satellite imagery, text, and environmental variables, the WildSAT method is proposed to perform contrastive learning on satellite image encoders.
WildSeg3D: Segment Any 3D Objects in the Wild from 2D Images
Yansong Guo (Xiamen University), Liujuan Cao (National University of Singapore)
CodeObject DetectionSegmentationImagePoint Cloud
π― What it does: We propose WildSeg3D, a feed-forward 3D object segmentation framework based on 2D images, capable of real-time segmentation of arbitrary 3D objects without scene-specific training.
World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model
Yupeng Zheng (Institute of Automation, Chinese Academy of Sciences), Dongbin Zhao (Tsinghua University)
CodeAutonomous DrivingTransformerVision Language ModelWorld ModelMultimodality
π― What it does: This paper proposes World4Drive, an intent-aware physical latent world model that achieves perception annotation-free end-to-end autonomous driving planning.
X2I: Seamless Integration of Multimodal Understanding into Diffusion Transformer via Attention Distillation
Jian Ma (OPPO AI Center), Zhenyu Yang (OPPO AI Center)
CodeGenerationData SynthesisKnowledge DistillationTransformerLarge Language ModelDiffusion modelImageVideoTextMultimodalityAudio
π― What it does: Proposes the X2I framework, which transfers the understanding capabilities of multimodal large language models to the Diffusion Transformer, enabling image generation and editing from multimodal inputs such as text, multiple languages, images, videos, and audio.
XTrack: Multimodal Training Boosts RGB-X Video Object Trackers
Yuedong Tan (University of Wurzburg), Radu Timofte (University of Wurzburg)
CodeObject TrackingTransformerMixture of ExpertsVideoMultimodality
π― What it does: This paper proposes a cross-modal knowledge sharing framework based on Mixture of Modal Experts to address the issues of data sparsity and modality isolation in multi-modal visual tracking.
π― What it does: This paper proposes FedYoYo, which addresses representation bias caused by heterogeneous and long-tailed data in federated learning through weak-strong enhanced self-teaching and distribution-aware Logit adjustment.
Zero-Shot Compositional Video Learning with Coding Rate Reduction
Heeseok Jung (Hanyang University), Eun-Sol Kim (Hanyang University)
CodeRecognitionCompressionTransformerVision Language ModelVideo
π― What it does: A zero-shot compositional video understanding framework based on the decline of coding rate in information theory is proposed, utilizing a resampling transformer to decouple spatiotemporal features and achieve predictions of unknown compositional actions.
Zero-Shot Vision Encoder Grafting via LLM Surrogates
Kaiyu Yue (University of Maryland), Tom Goldstein (University of Maryland)
CodeRecognitionCompressionComputational EfficiencyTransformerLarge Language ModelVision Language ModelMultimodality
π― What it does: This study proposes constructing a small language model surrogate and training a visual encoder on it, then directly integrating this encoder onto a large-scale LLM without additional training to achieve zero-shot visual language reasoning.
Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
Ziming Yu (Beijing Normal University), Hua Huang (Beijing Normal University)
CodeOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: We propose SubZero, a zero-order optimization framework based on hierarchical low-rank random subspaces for fine-tuning large-scale LLMs, which avoids backpropagation and significantly reduces memory usage.
π― What it does: A blind image super-resolution method based on diffusion models, ZFusion, is proposed, which utilizes synthetic zero-shot learning to estimate unknown degradations and embed them into the generation process.