IEEE/CVF International Conference on Computer Vision · 833 papers
SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions
Jessica Bader (Technical University of Munich), Zeynep Akata (Technical University of Munich)
CodeRecognitionGenerationData SynthesisTransformerVision Language ModelDiffusion modelImageBenchmark
🎯 What it does: This paper proposes the SUB (Synthetic Attribute Substitutions) benchmark dataset to evaluate the generalization ability of Concept Bottleneck Models (CBM) and Visual Language Models (VLM) when faced with fine-grained attribute substitutions, and generates high-quality attribute substitution images through a new method called Tied Diffusion Guidance (TDG);
🎯 What it does: The paper proposes using existing adaptive optics systems to achieve multi-image super-resolution by applying learnable phase distortions to the wavefront, thereby enhancing optical resolution while maintaining AO correction.
SVIP: Semantically Contextualized Visual Patches for Zero-Shot Learning
Zhi Chen (University of Southern Queensland), Zi Huang (University of Queensland)
CodeClassificationRecognitionTransformerImage
🎯 What it does: This paper proposes a zero-shot learning framework based on Vision Transformer, called SVIP, which addresses the semantic mismatch problem by identifying and processing semantically irrelevant image patches during the input stage.
SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition
Yongkun Du (Fudan University), Yu-Gang Jiang (Fudan University)
CodeRecognitionTransformerText
🎯 What it does: This paper presents SVTRv2, an improved scene text recognition model with a CTC structure that significantly enhances the handling of irregular text and language context while maintaining fast inference characteristics.
Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detection
Jinglun Li (Fudan University), Wenqiang Zhang (Fudan University)
CodeData SynthesisAnomaly DetectionTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringDiffusion modelImageBenchmark
🎯 What it does: By using a multimodal large language model to extract contextual labels, we iteratively generate near-boundary OOD samples in the image feature space using a diffusion model, and fine-tune the image encoder and text negative label features of CLIP with these synthetic samples, thereby improving OOD detection performance.
CodeGenerationData SynthesisTransformerLarge Language ModelAgentic AIPrompt EngineeringImageTextMultimodalityBenchmark
🎯 What it does: A training-free multi-agent system is proposed, which automatically parses prompts, selects models, and iteratively optimizes the text-to-image generation process.
TAB: Transformer Attention Bottlenecks enable User Intervention and Debugging in Vision-Language Models
Pooyan Rahmanzadehgervi (Auburn University), Anh Totti Nguyen (Auburn University)
CodeObject DetectionExplainability and InterpretabilityTransformerVision Language ModelImageText
🎯 What it does: A single-head Transformer Attention Bottleneck (TAB) layer was designed and implemented, inserted into a vision-language model for image difference description, enabling the model to provide both difference explanations and interpretable, editable attention.
🎯 What it does: Proposes the OTFM framework, which utilizes unbalanced optimal transport combined with flow matching to achieve high-quality image stitching in a single step.
Taming the Untamed: Graph-Based Knowledge Retrieval and Reasoning for MLLMs to Conquer the Unknown
Bowen Wang (Osaka University), Yuta Nakashima (Osaka University)
CodeRetrievalGraph Neural NetworkLarge Language ModelPrompt EngineeringMultimodalityGraphBenchmark
🎯 What it does: A multimodal knowledge graph themed on 'Monster Hunter: World' (MH-MMKG) was constructed, and a benchmark test consisting of 238 questions was designed. An untrained multi-agent retriever was proposed to allow MLLM to automatically acquire relevant knowledge from the graph, and then enhance reasoning to answer questions.
🎯 What it does: This paper proposes an online point tracking model called TAPNext, which transforms the task of tracking any point (TAP) into next token prediction, utilizing an interleaved structure of SSM and ViT for end-to-end training without specific tracking biases.
CodeGenerationData SynthesisTransformerLarge Language ModelAuto EncoderMesh
🎯 What it does: This paper proposes an autoregressive 3D asset generation framework based on a three-plane quantized variational autoencoder (3D VQ-VAE) and GPT, named TAR3D. It first compresses 3D meshes into discrete three-plane representations, and then uses a pre-trained GPT to progressively generate geometric components in a 'next-part' manner, resulting in high-quality 3D models.
🎯 What it does: This paper proposes a multi-modal foundation model-based audio-video segmentation framework called TAViS, which integrates ImageBind and SAM2, achieving alignment and prompting of audio, visual, and text through a text bridge.
🎯 What it does: A two-stage framework called TeethGenerator is proposed, capable of synthesizing paired 3D point clouds of orthodontic anterior and posterior teeth.
🎯 What it does: A multi-modal video fusion framework called TemCoCo is proposed, which takes into account visual quality, semantic accuracy, and temporal consistency.
🎯 What it does: A zero-shot depth completion method based on visual prompts is proposed under the conditions of having only RGB images and sparse depth measurements.
Test-Time Retrieval-Augmented Adaptation for Vision-Language Models
Xinqi Fan (Manchester Metropolitan University), Mubarak Shah (Chinese University of Hong Kong)
CodeRetrievalDomain AdaptationTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: A training-free Test-time Retrieval-Augmented Adaptation (TT-RAA) framework is proposed, which utilizes a Streaming Gaussian Mixture Database (SMGD) to estimate the test distribution in real-time and enhances the inference performance of CLIP through a Retrieval-Augmented Module (MRA).
CodeObject DetectionConvolutional Neural NetworkVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a semantic text-guided infrared small target detection framework called Text-IRSTD, which significantly improves target detection and contour recovery in complex scenes.
The Devil is in the Spurious Correlations: Boosting Moment Retrieval with Dynamic Learning
Xinyang Zhou (University of Electronic Science and Technology of China), Wen Li (University of Electronic Science and Technology of China)
CodeRetrievalTransformerVideoText
🎯 What it does: This study proposes a dynamic learning method to address the issue of 'spurious correlation' in video temporal retrieval, combining video synthesis and temporal dynamic enhancement to achieve more accurate text-video temporal matching.
The Inter-Intra Modal Measure: A Predictive Lens on Fine-Tuning Outcomes in Vision-Language Models
Laura Niss (MIT Lincoln Laboratory), Theodoros Tsiligkaridis (MIT Lincoln Laboratory)
CodeDomain AdaptationRepresentation LearningTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Proposes the Inter-Intra Modal Measure (IIMM) to predict the performance improvement and catastrophic forgetting of dual-encoded visual-language models after fine-tuning without requiring fine-tuning itself.
🎯 What it does: By constructing a large-scale noise library and retrieving initial noise that matches user objectives, we achieve training-free and interpretable target-driven image generation.
Think Twice: Test-Time Reasoning for Robust CLIP Zero-Shot Classification
Shenyu Lu (Purdue University), Xiaoqian Wang (Purdue University)
CodeClassificationObject DetectionTransformerVision Language ModelContrastive LearningImageMultimodalityBenchmark
🎯 What it does: This paper proposes an unsupervised test-time reasoning method called TTR, which utilizes CLIP's own semantic representation for object localization in images and performs zero-shot classification only within the localized areas to mitigate the impact of spurious correlations on CLIP's zero-shot classification.
TikZero: Zero-Shot Text-Guided Graphics Program Synthesis
Jonas Belouadi (University of Mannheim), Simone Ponzetto (University of Mannheim)
CodeGenerationData SynthesisKnowledge DistillationTransformerLarge Language ModelVision Language ModelTextMultimodality
🎯 What it does: This paper proposes Ti k Zero, a zero-shot text-driven TikZ graphic program synthesis method that achieves this by using visual representations as a bridge.
🎯 What it does: A lightweight visual Mamba backbone, TinyViM, is designed and implemented, utilizing a frequency-decoupled Laplace Mixer and a frequency gradient guidance (Inception) mechanism to handle low-frequency global context and high-frequency local details, achieving efficient global modeling and high-frequency retention.
TITAN: Query-Token based Domain Adaptive Adversarial Learning
Tajamul Ashraf (Mohamed bin Zayed University of Artificial Intelligence), Janibul Bashir (National Institute of Technology Srinagar)
CodeObject DetectionDomain AdaptationKnowledge DistillationTransformerImageBiomedical Data
🎯 What it does: A TITAN framework is proposed for source-free domain adaptive object detection, which divides target domain samples into easy/difficult subsets based on detection variance, and incorporates query-token adversarial learning in a student-teacher model to achieve domain alignment.
To Label or Not to Label: PALM - A Predictive Model for Evaluating Sample Efficiency in Active Learning Models
Julia Machnio (Pioneer Centre for AI University of Copenhagen), Mostafa Mehdipour Ghazi (Pioneer Centre for AI University of Copenhagen)
CodeExplainability and InterpretabilityComputational EfficiencyConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes the PALM model, which predicts sample efficiency and learning curves in the active learning process through interpretable mathematical parameters.
ToolVQA: A Dataset for Multi-step Reasoning VQA with External Tools
Shaofeng Yin (Peking University), Yang Liu (Peking University)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: A real-world multimodal tool usage question-answering dataset, ToolVQA, has been constructed, and a ToolEngine generation pipeline for automatically generating multi-step reasoning samples has been proposed.
Toward Fair and Accurate Cross-Domain Medical Image Segmentation: A VLM-Driven Active Domain Adaptation Paradigm
Hongqiu Wang (Hong Kong University of Science and Technology), Lei Zhu (Hong Kong University of Science and Technology)
CodeSegmentationDomain AdaptationTransformerVision Language ModelContrastive LearningImageMultimodalityMagnetic Resonance Imaging
🎯 What it does: This paper proposes a Fair Active Domain Adaptation (Fair-ADA) paradigm based on Visual-Language Models (VLM) for cross-domain medical image segmentation tasks, achieving both fairness and efficiency in model transfer with a limited number of labeled samples (approximately 5%). The method utilizes VLM to achieve semantic alignment between images and sensitive attributes (such as race and gender) and designs a FairAP sampling strategy that combines attributes with polysemy to ensure that balanced and diverse samples from various subgroups are included in the labeled set. The model trained on the source domain is then fine-tuned to obtain a fairer and better-performing target domain model.
Toward Material-Agnostic System Identification from Videos
Yizhou Zhao (Carnegie Mellon University), Min Xu (Carnegie Mellon University)
CodeOptimizationGaussian SplattingVideo
🎯 What it does: The MASIV (Material-Agnostic System Identification from Videos) framework is proposed, which utilizes multi-view videos to achieve material-agnostic identification of object geometry and dynamics through dynamic Gaussian reconstruction, continuous particle trajectory inference, and a differentiable MPM combined with a neural constitutive model.
Towards a 3D Transfer-based Black-box Attack via Critical Feature Guidance
Shuchao Pang (Nanjing University of Science and Technology), Yongbin Zhou (Nanjing University of Science and Technology)
CodeClassificationAdversarial AttackPoint Cloud
🎯 What it does: This paper proposes a 3D point cloud black-box transfer attack method based on key feature guidance, named CFG, which can generate adversarial point clouds with a high transfer rate without any information about the target model.
🎯 What it does: The Copernicus-Pretrain dataset, Copernicus-FM model, and Copernicus-Bench benchmark are proposed, and the effectiveness of the unified base model for surface and atmospheric multi-tasking is validated.
🎯 What it does: A method is proposed to enhance the generalization of 3D medical multimodal tasks through learning personalized invariant representations.
Towards Accurate and Efficient 3D Object Detection for Autonomous Driving: A Mixture of Experts Computing System on Edge
Linshen Liu (Johns Hopkins University), Hao Frank Yang (Johns Hopkins University)
CodeObject DetectionAutonomous DrivingComputational EfficiencyMixture of ExpertsMultimodalityPoint Cloud
🎯 What it does: A marginal 3D object detection system EMC2 based on Mixture of Experts (MoE) is proposed, utilizing multimodal fusion of LiDAR and cameras along with scene-adaptive expert scheduling to enhance autonomous driving perception performance.
🎯 What it does: An absorptive adaptation learning framework is proposed for extreme cross-domain few-shot learning, guiding the base model to absorb knowledge through an expert model.
🎯 What it does: The ToMiE method is proposed, which adaptively grows a tree on the SMPL skeleton to accurately model 3D human figures with handheld objects and loose clothing, and supports explicit animation.
Towards Higher Effective Rank in Parameter-Efficient Fine-tuning using Khatri-Rao Product
Paul Albert (Amazon Machine Learning), Ehsan Abbasnejad (Monash University)
CodeTransformerSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: A parameter-efficient fine-tuning method based on the Khatri-Rao product, called KRAdapter, is proposed to address the low-rank limitation of LoRA.
Towards Human-like Virtual Beings: Simulating Human Behavior in 3D Scenes
Chen Liang (Zhejiang University), Yi Yang (Zhejiang University)
CodeRobotic IntelligenceTransformerLarge Language ModelAgentic AI
🎯 What it does: Designed and implemented an autonomous intelligent agent ACTOR based on large language models (LLM), capable of executing high-level, long-term abstract goal-driven behaviors in real 3D home environments, and achieving dynamic environment adaptation through a perception-planning-execution loop. A large-scale scene perception and action-rich BEHAVIORHUB dataset was also constructed for training and evaluating such behavior simulations.
🎯 What it does: This paper proposes the Skip-DiT architecture, which incorporates long skip connections into the Diffusion Transformer (DiT) to enhance feature stability and achieve efficient caching acceleration.
🎯 What it does: A TR-PTS framework is proposed, achieving efficient fine-tuning in the ViT model through task-related parameter selection and token selection.
🎯 What it does: This work proposes the TRACE framework, which utilizes multi-view RGB videos for unsupervised learning of 3D Gaussian particles as rigid body particles, directly predicting their translational-rotational dynamic parameters, thereby achieving future frame extrapolation.
🎯 What it does: A large-scale infrared small drone tracking dataset TDTIV has been constructed, and a Motion-Centric Adaptive Tracker (MCATrack) has been proposed for tracking very small targets.
🎯 What it does: This paper constructs a large-scale, object-centered video dataset called TrackVerse and proposes a mutation-aware contrastive learning framework based on data augmentation parameters to achieve unsupervised representation learning of object state changes.
Trade-offs in Image Generation: How Do Different Dimensions Interact?
Sicheng Zhang (Khalifa University), Zhichao Lu (City University of Hong Kong)
CodeGenerationTransformerVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes TRIG-Bench, a multi-dimensional evaluation benchmark covering three major tasks: text-to-image, image editing, and topic generation, with 10 dimensions and 132 dimension pairs.
🎯 What it does: A training-free category purification framework called FreeCP is proposed to enhance the accuracy of open vocabulary semantic segmentation.
🎯 What it does: A training-free diffusion model is proposed for geometric image editing, which separates object transformation, source area restoration, and target area refinement.
🎯 What it does: A TF-IDG framework is proposed without the need for any model training, achieving diverse, fine-grained, and structurally consistent industrial defect synthesis through image alignment, mask guidance, and texture preservation.
Training-Free Text-Guided Image Editing with Visual Autoregressive Model
Yufei Wang (Snap Research), Jian Wang (Nanyang Technological University)
CodeImage TranslationGenerationVision Language ModelImageBenchmark
🎯 What it does: This paper proposes a training-free text-guided image editing framework called AREdit, based on a Visual Autoregressive Model (VAR), which achieves high-fidelity local edits without explicit inversion while keeping other areas of the original image unchanged.
Tree Skeletonization from 3D Point Clouds by Denoising Diffusion
Elias Ariel Marks (University of Bonn), Cyrill Stachniss (University of Bonn)
CodeSegmentationGenerationDiffusion modelPoint CloudAgriculture Related
🎯 What it does: This paper proposes a trunk skeleton reconstruction method based on a denoising diffusion probabilistic model, which can predict the positions and flow directions of branch nodes from partially occluded colored point clouds, and constructs a complete trunk skeleton using a minimum spanning tree algorithm.
🎯 What it does: This paper presents TurboTrain, an efficient and balanced multi-task learning framework for end-to-end training of multi-agent perception and prediction.
CodeRecognitionSegmentationPose EstimationDepth EstimationSimultaneous Localization and MappingImageMultimodalityPoint CloudBenchmark
🎯 What it does: This paper presents UAVScenes—a large-scale, multi-modal drone dataset that provides frame-level image and LiDAR point cloud semantic annotations, 6 degrees of freedom (DoF) poses, and 3D maps, and conducts benchmark experiments for multiple tasks (image/point cloud semantic segmentation, depth estimation, localization, scene recognition, NVS, etc.).
🎯 What it does: This paper designs a dual-camera system based on a beam splitter and aligns it with DFT to construct the first real-world UDC video dataset, UDC-VIT, and evaluates various deep learning models based on this dataset.
UIPro: Unleashing Superior Interaction Capability For GUI Agents
Hongxin Li (University of Chinese Academy of Sciences), Zhaoxiang Zhang (University of Chinese Academy of Sciences)
CodeRobotic IntelligenceTransformerSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: A large-scale dataset for GUI understanding and interaction has been constructed, and a unified action space has been established, resulting in the training of a general GUI agent UIPro that can perform clicks, inputs, scrolling, and other operations across multiple platforms.
🎯 What it does: A unified multi-domain adaptive tracking framework UMDATrack is proposed to maintain high-quality target state prediction under various adverse weather conditions.
Uncertainty-Aware Diffusion-Guided Refinement of 3D Scenes
Sarosij Bose (University of California, Riverside), Amit K. Roy-Chowdhury (University of California, Riverside)
CodeRestorationSegmentationGenerationDepth EstimationLarge Language ModelDiffusion modelVideoPoint Cloud
🎯 What it does: This paper presents UAR-Scenes, a diffusion-guided 3D scene refinement pipeline based on uncertainty perception, which can further enhance the rendering quality of new viewpoints on rough point clouds generated from single-view to 3D reconstruction models (such as Flash3D), especially for reasonable completion of unobserved areas.
Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models
Xiao Liang (Xidian University), Tat-Seng Chua (National University of Singapore)
CodeRecognitionRetrievalTransformerVision Language ModelMultimodalityBiomedical DataMagnetic Resonance Imaging
🎯 What it does: The Expert-CFG framework is proposed, which enhances the reliability of medical vision-language models without training by utilizing uncertainty estimation and expert annotations.
🎯 What it does: This paper proposes a lightweight deep unfolded joint encoding-decoding network (UED-Net) that achieves high-quality fusion of remote sensing images by recursively encoding multi-level spatial-spectral degraded features and aggregating information at various stages.
🎯 What it does: A unified adversarial augmentation framework (UAA) is proposed, which generates adversarial palm images with both geometric distortion and texture degradation through spatial transformation and identity-preserving generative networks, thereby enhancing the robustness of palm recognition models on complex samples.
🎯 What it does: A unified open-world segmentation model COSINE is proposed, capable of performing various tasks such as semantic, instance, panoptic, referring, and video object segmentation through text and image multimodal prompts.
🎯 What it does: The UniFuse framework is proposed to achieve integrated processing of multimodal medical images in three tasks: distortion, misalignment, and fusion.
🎯 What it does: The UniGS model is proposed, which uses a unified world space 3D Gaussian set for sparse view 3D reconstruction and novel view synthesis.
🎯 What it does: For inconsistent multi-view images, we first use a Video Diffusion Model (VDM) to restore the images to a consistent state, and then utilize methods such as NeRF/3DGS for 3D reconstruction.
🎯 What it does: A method for Unknown Text Learning (UTL) is proposed, utilizing the CLIP model to simultaneously learn unknown text and contextual prompts in few-shot open-set recognition tasks.
Unlearning the Noisy Correspondence Makes CLIP More Robust
Haochen Han (Peng Cheng Laboratory), Fangming Liu (Peng Cheng Laboratory)
CodeClassificationRetrievalTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: Fine-tuning the pre-trained CLIP model using 'hardest negative samples' to achieve 'unlearning' of noise pairs (FP and FN), thereby enhancing the model's robustness to noise correspondence.
🎯 What it does: A training-independent Levenberg-Marquardt-Langevin (LML) method is proposed, which approximates the Hessian of the diffusion model through low-rank approximation and damping techniques, and uses it for Langevin updates to improve sampling quality.
🎯 What it does: Proposes the FlashVDM framework, which accelerates the pre-trained Vecset Diffusion Model (VDM) to 5 steps through Progressive Flow Distillation and an efficient VAE decoder, achieving high-quality 3D shape generation within 1 second;
🎯 What it does: A source-agnostic panoramic occlusion-free seamless segmentation task is proposed, and the UNLOCK framework is designed to achieve model adaptation in the target panoramic domain.
UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI
Fangwei Zhong (Beijing Normal University), Yizhou Wang (Peking University)
CodeRobotic IntelligenceLarge Language ModelReinforcement LearningVision Language ModelImageMultimodality
🎯 What it does: The UnrealZoo platform has been constructed, providing 100 high-fidelity virtual worlds based on Unreal Engine and 67 types of playable entities. It achieves efficient data collection and distributed training for evaluating embodied AI tasks such as visual navigation and tracking through the optimization of the UnrealCV+ and Gym interface.
🎯 What it does: This paper proposes a method that utilizes a collection of unpaired clean images and corrupted images to learn the forward degradation operator A in image inverse problems through a diffusion model and Conditional Flow Matching (CFM). After obtaining the degradation operator, a non-blind method is used to restore the image. Additionally, applications to non-uniform blur, camera lens distortion, and single-image super-resolution are demonstrated.
🎯 What it does: Proposes an unsupervised learning framework that utilizes a single network to simultaneously estimate the optical flow (motion) and image intensity (appearance) of event cameras.
🎯 What it does: An unsupervised part discovery framework called Mask Part Autoencoder (MPAE) is proposed, which aligns part descriptors using low-level visual features by filling in part descriptors and recovering images on randomly occluded images, resulting in pixel-level segmentation that closely matches the true part shapes.
🎯 What it does: A Mapping and Collaborative Learning (MCL) framework is proposed for training unsupervised visible-infrared person re-identification models under unpaired conditions.
UPRE: Zero-Shot Domain Adaptation for Object Detection via Unified Prompt and Representation Enhancement
Xiao Zhang (Dalian University of Technology), Xiangxiang Chu (Alibaba Group)
CodeObject DetectionDomain AdaptationPrompt EngineeringVision Language ModelImage
🎯 What it does: This paper proposes a framework for Unified Prompt and Representation Enhancement (UPRE) to achieve zero-shot domain adaptation for object detection under unlabelled target domain images.
UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence
Jie Feng (Tsinghua University), Yong Li (Tsinghua University)
CodeTransformerLarge Language ModelVision Language ModelImageMultimodalityBenchmarkChain-of-Thought
🎯 What it does: This paper proposes UrbanLLaVA, which unifies the processing of four types of urban multimodal data (street view images, satellite images, spatial data, and trajectory data) and achieves cross-modal reasoning and decision-making in multi-city tasks.
VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting
Hao Chen (Hong Kong University of Science and Technology), Lei Bai (Shanghai AI Laboratory)
CodeTransformerMixture of ExpertsTime Series
🎯 What it does: A Variable Adaptive Mixture of Experts (VA-MoE) framework is proposed for incremental weather forecasting, which supports the dynamic addition of new meteorological variables without retraining the entire model.
🎯 What it does: This paper proposes Vector Contrastive Learning and the COVER framework to address the issue of over-dispersion in pixel-level pre-training of medical images.
Verbalized Representation Learning for Interpretable Few-Shot Generalization
Cheng-Fu Yang (University of California), Kai-Wei Chang (University of California)
CodeClassificationExplainability and InterpretabilityRepresentation LearningTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: A Verbalized Representation Learning (VRL) method is proposed, which utilizes visual-language models (VLM) to automatically generate interpretable natural language features from a small number of samples and maps them to numerical vectors for downstream classification tasks.
🎯 What it does: This paper proposes the ViCTr two-stage framework for generating medical images that balance anatomical integrity and pathological details, particularly for the synthesis of abdominal CT/MRI and liver cirrhosis MRI.
Vid-Group: Temporal Video Grounding Pretraining from Unlabeled Videos in the Wild
Peijun Bao (Nanyang Technological University), Alex Kot (Nanyang Technological University)
CodeRecognitionRetrievalTransformerLarge Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: A massive video temporal annotation dataset, Vid-Group, was constructed without the need for manual labeling, and the ReCorrect algorithm was proposed, which significantly improves the performance of Temporal Video Grounding (TVG) during the pre-training phase through self-correction (semantic-guided refinement + memory consistency correction).
CodeImage TranslationImage HarmonizationGenerationLarge Language ModelDiffusion modelVideoText
🎯 What it does: A color grading framework based on reference videos is proposed, which explicitly generates LUTs using diffusion models and adjusts user preferences through text prompts.
🎯 What it does: For crowded scenes captured by high-speed moving drones, a Video Individual Counting (VIC) method is proposed, which uses a shared density map-guided deep cross-frame attention network to directly estimate the shared density map and derive the inflow and outflow density maps, ultimately counting the unique individuals in the video.
🎯 What it does: Proposes the Video2BEV scheme, which converts drone videos into bird's-eye views (BEV) to enhance cross-platform geographic positioning accuracy.
VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridges
Yuxuan Wang (State Key Laboratory of General Artificial Intelligence), Zilong Zheng (State Key Laboratory of General Artificial Intelligence)
CodeSegmentationRetrievalComputational EfficiencyTransformerLarge Language ModelVision Language ModelVideoText
🎯 What it does: This paper proposes the VideoLLaMB framework, which achieves efficient understanding of long video streams using a recursive memory bridge layer and scene chunking algorithm, and can generate video question-answering and planning results in real-time on a single GPU.
🎯 What it does: VIGFace proposes a method for pre-allocating virtual identities in the feature space and generating privacy-friendly synthetic facial images using diffusion models.
🎯 What it does: This paper studies a two-stage monocular semantic scene completion framework called VisHall3D, which first recovers visible areas and then infers invisible areas.
Vision-Language Interactive Relation Mining for Open-Vocabulary Scene Graph Generation
Yukuan Min (Xidian University), Cheng Deng (Xidian University)
CodeObject DetectionGenerationTransformerVision Language ModelImageTextMultimodality
🎯 What it does: Proposes the Vision-Language Interactive Relation Mining model (VL-IRM), which achieves open vocabulary scene graph generation through interactive learning between vision and text.
🎯 What it does: A high-resolution video inverse problem solving framework utilizing the latent diffusion model (SDXL) is proposed, supporting various spatiotemporal degradations (deblurring, super-resolution, inpainting, and their combinations with frame averaging) and achieving efficient real-time reconstruction on a single GPU.
VisNumBench: Evaluating Number Sense of Multimodal Large Language Models
Tengjin Weng (Shenzhen University), Zhong Ming (Shenzhen University)
CodeTransformerLarge Language ModelVision Language ModelImageMultimodalityBenchmark
🎯 What it does: A benchmark called VisNumBench is proposed to evaluate the intuitive numerical sense capabilities of multimodal large models in visual numerical sense.
VISO: Accelerating In-orbit Object Detection with Language-Guided Mask Learning and Sparse Inference
Meiqi Wang (Tsinghua University), Han Qiu (Tsinghua University)
CodeObject DetectionVision Language ModelImageText
🎯 What it does: The VISO model is proposed, which combines language-guided mask learning and sparse inference to achieve high-precision detection of small targets in satellite orbital environments and significantly improve inference speed.
ViSpeak: Visual Instruction Feedback in Streaming Videos
Shenghao Fu (Sun Yat-sen University), Wei-Shi Zheng
CodeLarge Language ModelSupervised Fine-TuningVision Language ModelVideoMultimodalityBenchmarkAudio
🎯 What it does: A new task called Visual Instruction Feedback is proposed, which requires the model to actively recognize and respond to visual gestures and commands in streaming video.
VisRL: Intention-Driven Visual Perception via Reinforced Reasoning
Zhangquan Chen (Tsinghua University), Dongsheng Li (Microsoft Research Asia)
CodeTransformerSupervised Fine-TuningReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: Proposes the VisRL framework, which utilizes reinforcement learning (step-level DPO) to achieve intention-driven visual perception and reasoning through self-evolution via task rewards without any bounding box annotations.
Visual Intention Grounding for Egocentric Assistants
Pengzhan Sun (National University of Singapore), Angela Yao (National University of Singapore)
CodeObject DetectionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a task of inferring user intentions from a first-person perspective and locating corresponding target objects in a scene. A large-scale dataset called EgoIntention is constructed, and based on this, instruction fine-tuning of multimodal large language models is performed.
Tiange Luo (University of Michigan), Honglak Lee (University of Michigan)
CodeVision Language ModelImage
🎯 What it does: This paper proposes RegionFocus, a method for dynamically focusing on GUI sub-regions during interaction to enhance the localization accuracy of visual language model agents.
Visual-Oriented Fine-Grained Knowledge Editing for MultiModal Large Language Models
Zhen Zeng (Hefei University of Technology), Meng Wang (Hefei University of Technology)
CodeKnowledge DistillationTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: Proposes a vision-guided fine-grained multimodal knowledge editing task, constructs the FGVEdit evaluation benchmark, and designs the Multimodal Scope Classifier-based Knowledge Editor (MSCKE) framework to implement this task.
Ziyu Liu (Shanghai Jiao Tong University), Jiaqi Wang (Shanghai Jiao Tong University)
CodeClassificationObject DetectionTransformerSupervised Fine-TuningReinforcement LearningVision Language ModelImageMultimodality
🎯 What it does: This paper proposes the Visual-RFT method, which applies verifiable rewards for reinforcement fine-tuning on large-scale visual language models (LVLM), enabling the model to enhance its visual reasoning and perception capabilities through self-trial and error with limited data.
ViT-EnsembleAttack: Augmenting Ensemble Models for Stronger Adversarial Transferability in Vision Transformers
Hanwen Cao (Huazhong University of Science and Technology), Kun He (Huazhong University of Science and Technology)
CodeAdversarial AttackTransformerImage
🎯 What it does: This paper proposes ViT-EnsembleAttack, an ensemble adversarial attack method targeting Vision Transformers, which enhances attack transferability through model adversarial augmentation.