IEEE/CVF International Conference on Computer Vision Β· 833 papers
Multimodal LLMs as Customized Reward Models for Text-to-Image Generation
Shijie Zhou (University at Buffalo), Changyou Chen (University at Buffalo)
CodeGenerationRecommendation SystemTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
π― What it does: A multimodal reward model named LLaVA-Reward was designed and trained to evaluate the performance of text-to-image generation across multiple dimensions, including alignment, authenticity, aesthetics, and safety.
π― What it does: This paper proposes the music grounding task MGSV, which aims to locate suitable background music segments for short videos in a music library.
π― What it does: An efficient video quality assessment framework MVQA based on the Mamba state space model is designed and implemented, and a Unified Semantic and Distortion Sampling (USDS) method is proposed, which integrates low-resolution semantic information with high-resolution distortion details.
π― What it does: This paper proposes NavMorph, a self-evolving world model framework for visual-language navigation (VLN-CE) tasks in continuous environments, capable of dynamically updating environmental representations and performing proactive action planning during online testing.
Amirhossein Ansari (Simon Fraser University), Pulei Xiong (National Research Council Canada)
CodeAnomaly DetectionTransformerLarge Language ModelVision Language ModelImage
π― What it does: This paper proposes an improved zero-shot OOD detection framework called NegRefine, which utilizes negative label filtering and multi-label matching scoring to achieve a more robust distinction between in-distribution and OOD.
π― What it does: Proposes Neighborhood Autoregressive Modeling (NAR), treating visual generation as a step-by-step outpainting process starting from an initial token, using a 'next neighbor prediction' mechanism that progresses from near to far.
π― What it does: An iterative tracking framework named NETracer is proposed, which utilizes a node-edge estimation network and a geographic distance search strategy to accurately extract tubular structures in images.
Siyu Ren (City University of Hong Kong), Wenping Wang (Texas A&M University)
CodeCompressionPoint CloudMesh
π― What it does: This paper proposes the NeCGS framework for compressing collections of multi-class 3D mesh models, which can maintain detail at extremely high compression ratios.
π― What it does: This paper proposes a phase-shift structured light measurement method that integrates displacement fields into a multi-camera single-projector neural inverse rendering framework for high-precision 3D measurement of moving objects.
π― What it does: An end-to-end neural inverse rendering framework is proposed, capable of simultaneously recovering geometry, spatially varying BRDF, and light source direction and intensity from multi-view single-light images.
π― What it does: This paper proposes a model NMR-KAN that combines Neuromanifold regularization with Kolmogorov-Arnold Networks (KAN), aiming to enhance the shape bias of visual models and suppress reliance on texture, thereby improving the model's robustness against image distortion and adversarial attacks.
Neurons: Emulating the Human Visual Cortex Improves Fidelity and Interpretability in fMRI-to-Video Reconstruction
Haonan Wang (Hong Kong University of Science and Technology), Xiaomeng Li (Hong Kong University of Science and Technology)
CodeRecognitionSegmentationGenerationExplainability and InterpretabilityLarge Language ModelDiffusion modelContrastive LearningVideoMultimodalityMagnetic Resonance Imaging
π― What it does: By simulating the hierarchical functions of the human visual cortex, the NEURONS framework is proposed, which breaks down the fMRI-to-video reconstruction task into four sub-tasks: key object segmentation, concept recognition, scene description, and fuzzy video reconstruction. The outputs of these sub-tasks are used as conditional guidance for a text-to-video diffusion model to achieve high-quality video reconstruction.
π― What it does: A diffusion model guided by neural operators (NeurOp-Diff) is proposed to achieve continuous scale super-resolution of remote sensing images.
π― What it does: A method for clothing deformation based on neural gradient fields is proposed by recovering high-fidelity geometry and materials of dynamic clothing from monocular videos, and detail capture is achieved geometrically with adaptive remeshing.
π― What it does: This paper proposes NoiseController, a noise control framework for multi-view video generation, which enhances the spatiotemporal consistency of videos through multi-level noise decomposition, multi-frame noise collaboration, and joint denoising.
π― What it does: Transforming the cross-view segmentation problem into an object mask matching task, using FastSAM to generate candidate masks, and matching source masks and candidate masks through contrastive learning.
Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis
Letian Zhang (Tongji University), Cheng Yang (Bytedance)
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageMultimodality
π― What it does: Proposes the Oasis method, which generates high-quality multimodal instruction data through MLLM autoregression using a single image and filters it through four-dimensional quality assessment.
π― What it does: A zero-shot tuning method named ObjectMate is proposed, which utilizes the prior of object repetition to synthesize multi-view reference objects with scene descriptions into images, achieving high-fidelity identity preservation and lighting matching.
π― What it does: This paper proposes OcRFDet, which enhances the geometric feature representation and foreground modeling of multi-view 3D object detection through a dual-module approach of Object-Centered Radiance Field (OcRF) and Height-Aware Transparency Attention (HOA).
π― What it does: A comprehensive OOD performance prediction benchmark, ODP-Bench, is proposed, covering 29 different types of OOD datasets, 10 mainstream performance prediction algorithms, and publicly releasing 1444 pre-trained models to form a unified evaluation environment.
π― What it does: The Omegance method is proposed, which controls the detail granularity of generated images through a noise scaling parameter Ο during the reverse sampling process of diffusion models, enabling detail control on global, spatial (mask), and temporal (scheduling) levels;
π― What it does: Design and implement OminiControl, a minimalist image control framework that utilizes the existing VAE encoder and Transformer blocks of the Diffusion Transformer, supporting various control tasks with spatial alignment and non-alignment.
π― What it does: A training-free universal virtual try-on framework called OmniVTON is proposed, which can seamlessly synthesize clothing images onto target portraits in both indoor and outdoor scenes without training, and supports multi-person try-on.
On Large Multimodal Models as Open-World Image Classifiers
Alessandro Conti (University of Trento), Elisa Ricci (Fondazione Bruno Kessler)
CodeClassificationTransformerLarge Language ModelVision Language ModelImageMultimodality
π― What it does: Evaluate the image classification capability of large-scale multimodal models (LMM) under unrestricted categories (open world) and propose four comprehensive evaluation metrics.
On the Complexity-Faithfulness Trade-off of Gradient-Based Explanations
Amir Mehrpanah (KTH Royal Institute of Technology), Hossein Azizpour (KTH Royal Institute of Technology)
CodeExplainability and InterpretabilityConvolutional Neural NetworkImage
π― What it does: A unified spectral framework is proposed to quantify the trade-off between the complexity and fidelity of gradient explanations, and the complexity of explanations is reduced by controlling the high-frequency information of ReLU.
On the Generalization of Representation Uncertainty in Earth Observation
Spyros Kondylatos (National Observatory of Athens), Ioannis Papoutsis (National Technical University of Athens)
CodeClassificationSegmentationTransformerImage
π― What it does: In this paper, the authors pre-trained a representation uncertainty module on a large-scale Earth Observation (EO) dataset and conducted zero-shot evaluation on downstream EO tasks such as multi-label classification and semantic segmentation, proposing an uncertainty assessment framework and metrics suitable for multi-label and segmentation tasks.
π― What it does: This paper proposes replacing the traditional CNN image encoder with a Fourier Neural Operator (FNO) encoder to learn the underlying PDE dynamics of the environment in model-free visual reinforcement learning and to directly extract features from images.
Liang Chen (Mohammed Bin Zayed University of Artificial Intelligence), Zhiqiang Shen (Mohammed Bin Zayed University of Artificial Intelligence)
CodeClassificationDomain AdaptationComputational EfficiencyTransformerSupervised Fine-TuningVision Language ModelImageMultimodality
π― What it does: A lightweight attention adapter (RAda) based on Rational Matrix is proposed, which directly adjusts the final fusion representation of VLM through adaptive masking, achieving efficient fine-tuning of the multimodal decision process.
CodeAdversarial AttackVision Language ModelContrastive LearningImageTextBenchmark
π― What it does: This paper proposes the CrossVLAD benchmark for cross-task adversarial attack evaluation and designs the CRAFT framework for region-token alignment to achieve a single perturbation that deceives multiple downstream tasks on a unified vision-language model.
One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Models
Hao Fang (Tsinghua University), Ke Xu (Tsinghua University)
CodeRetrievalAdversarial AttackTransformerVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: This paper proposes a general adversarial perturbation generation framework C-PGC based on contrastive learning, which generates adversarial perturbations for images and texts in one go to deceive visual-language pre-trained models.
One Polyp Identifies All: One-Shot Polyp Segmentation with SAM via Cascaded Priors and Iterative Prompt Evolution
Xinyu Mao (Chinese University of Hong Kong), Max Meng (Southern University of Science and Technology)
CodeSegmentationTransformerPrompt EngineeringImageBiomedical Data
π― What it does: A single-sample polyp segmentation framework based on SAM, OP-SAM, is proposed, which automatically generates prompt points from a single annotated image, achieving zero training and extremely low annotation costs.
CodeObject DetectionSegmentationTransformerLarge Language ModelAgentic AIVideoBenchmark
π― What it does: A proxy-based framework is proposed, utilizing real-time digital twins to achieve online video inference segmentation, supporting multi-step semantic, spatial, and temporal reasoning.
ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
Zifu Wan (Carnegie Mellon University), Yaqi Xie (Carnegie Mellon University)
CodeRecognitionGenerationComputational EfficiencyTransformerVision Language ModelMultimodality
π― What it does: This paper proposes ONLY, a training method that requires only a single-layer intervention and no additional queries, aimed at significantly reducing the hallucination problem in large visual language models.
π― What it does: This paper proposes the Open Set Cross-Modal Generalization (OSCMG) task and designs the MICU framework to achieve unified representation while enhancing recognition performance for unknown categories.
π― What it does: Proposed the Adversarial Open Unfairness Discovery and Mitigation Network (AdvOU), which embeds a lightweight Unfairness Regulator in the deepfake detection model and alternately trains the OUD and UAM modules to achieve dynamic identification and elimination of undefined biases.
Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibration
Ting Lei (Wangxuan Institute of Computer Technology, Peking University), Yang Liu (Wangxuan Institute of Computer Technology, Peking University)
CodeRecognitionObject DetectionTransformerLarge Language ModelPrompt EngineeringContrastive LearningMultimodality
π― What it does: This paper proposes an open vocabulary human-object interaction (HOI) detection framework called INP-CC, which utilizes Interaction-Aware Prompts and Concept Calibration to address the distribution bias and semantic alignment issues in traditional CLIP for region-level interaction recognition.
π― What it does: An open-source code library for animal re-identification, OpenAnimals, has been constructed, and based on it, the applicability of various human re-identification methods to animal data has been re-evaluated, ultimately proposing a benchmark model specifically for animal re-identification, ARBase.
π― What it does: A multi-modal prompt-based remote sensing object detection framework called OpenRSD is proposed, balancing fast inference and high accuracy.
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining
Ming Hu (Shanghai AI Laboratory), Zongyuan Ge (Monash University)
CodeRecognitionObject DetectionRetrievalTransformerLarge Language ModelVision Language ModelContrastive LearningVideoTextMultimodalityRetrieval-Augmented Generation
π― What it does: This paper develops the OphCLIP visual-language pre-training framework and the OphVL large-scale hierarchical structure ophthalmic surgery video-text dataset to enhance phase recognition and multi-tool detection in ophthalmic surgery.
OracleFusion: Assisting the Decipherment of Oracle Bone Script with Structurally Constrained Semantic Typography
Caoshuo Li (Xiamen University), Rongrong Ji (Anyang Normal University)
CodeGenerationTransformerLarge Language ModelDiffusion modelTextMultimodality
π― What it does: The OracleFusion two-stage semantic typography framework is proposed to assist in interpreting oracle bone script. It first uses a multimodal large language model to analyze the glyphs of oracle bone characters and locate key components, and then employs a structure-constrained vector fusion method to generate semantically rich and structurally complete vector fonts.
CodeRecognitionObject DetectionGenerationTransformerVision Language ModelTextMultimodality
π― What it does: Proposes the OURO self-starting framework, which generates self-annotated data through recursive sub-region descriptions and VQA to train models, enhancing multimodal scene understanding;
OV-SCAN: Semantically Consistent Alignment for Novel Object Discovery in Open-Vocabulary 3D Object Detection
Adrian Chow (University of Waterloo), Krzysztof Czarnecki (University of Waterloo)
CodeObject DetectionAutonomous DrivingTransformerPrompt EngineeringVision Language ModelPoint Cloud
π― What it does: The OV-SCAN framework is proposed, utilizing the SC-NOD module to generate high-quality 3D annotations and filtering low-quality 2D-3D alignment pairs through selective alignment, combined with H2SA two-stage hierarchical alignment to achieve open vocabulary 3D object detection.
p-MoD: Building Mixture-of-Depths MLLMs via Progressive Ratio Decay
Jun Zhang (Nanjing University), Limin Wang (Nanjing University)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
π― What it does: The p-MoD architecture is proposed, utilizing the Mixture-of-Depths mechanism to layer-wise filter visual tokens in the Transformer layer, significantly reducing the computational cost of multimodal large language models.
PanoLlama: Generating Endless and Coherent Panoramas with Next-Token-Prediction LLMs
Teng Zhou (Zhejiang University), Yongchuan Tang (Zhejiang University)
CodeGenerationData SynthesisTransformerLarge Language ModelText
π― What it does: Through autoregressive models and token redirection, PanoLlama is proposed to achieve continuous generation of untrained, infinitely long panoramic images.
Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflection
Romain Thoreau (National Centre for Space Studies), Dawa Derksen (National Centre for Space Studies)
CodeClassificationSegmentationTransformerImage
π― What it does: This paper proposes the DEFLECT method, which achieves parameter-efficient adaptation of RGB pre-trained geospatial foundation models (GFM) on multispectral satellite images.
π― What it does: The researchers proposed a more realistic scenario in the text-point cloud cross-modal localization task, allowing for partial matching between text and submaps, and improved localization accuracy through uncertainty modeling and propagation.
π― What it does: This paper proposes PatchScaler, a single-image super-resolution method based on diffusion models, which can adaptively allocate sampling steps for different image patches and utilize texture prompts to enhance detail recovery.
π― What it does: A diffusion model called PathDiff has been developed to jointly generate pathological images from unpaired diagnostic texts and spatial masks.
PBFG: A New Physically-Based Dataset and Removal of Lens Flares and Glares
Jie Zhu (Sungkyunkwan University), Sungkil Lee (Sungkyunkwan University)
CodeRestorationData SynthesisTransformerImage
π― What it does: A PBFG physical benchmark glare and glare dataset is proposed, and based on this, the FGRNet network is trained to remove lens glare and glare in nighttime photography.
Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models
Hongyang Wei (Tsinghua University), Lei Zhang (Hong Kong Polytechnic University)
CodeRestorationSuper ResolutionTransformerLarge Language ModelGenerative Adversarial NetworkImage
π― What it does: Developed the Real-ISR framework PURE based on the autoregressive multimodal generative model (Lumina-mGPT), which can perceive the degradation level of low-quality images, understand their semantic content, and achieve high-quality image generation and restoration within a single model.
π― What it does: Proposes PerLDiff, which achieves controllable street scene image generation based on 3D annotations through a perspective layout diffusion model.
Personalized Federated Learning under Local Supervision
Qiqi Liu (Westlake University), Han Yu (Sony AI)
CodeFederated LearningImage
π― What it does: This paper proposes a personalized federated learning framework FedSimSup based on a local supervisor's parallel structure to improve model performance under non-IID conditions.
π― What it does: This paper proposes a Perspective-Aware Teacher (PAT) framework for knowledge distillation across different architectures (CNN, ViT, MLP).
CodeRestorationDomain AdaptationConvolutional Neural NetworkImagePhysics Related
π― What it does: Utilizing a physics-guided haze transfer network (PHATNet) for haze pattern migration between the source domain and the target domain, generating domain-specific fine-tuning datasets to achieve adaptive enhancement for real-time image dehazing.
Physical Degradation Model-Guided Interferometric Hyperspectral Reconstruction with Unfolding Transformer
Yuansheng Li (Beijing Institute of Technology), Ying Fu (Beijing Institute of Technology)
CodeRestorationData SynthesisTransformerImage
π― What it does: This paper proposes an interference-based hyperspectral imaging reconstruction method based on a physical attenuation modelβIHRUTβand generates realistic synthetic data using this model.
Pi-GPS: Enhancing Geometry Problem Solving by Unleashing the Power of Diagrammatic Information
Junbo Zhao (Beijing Normal University), Hua Huang (Beijing Normal University)
CodeLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: Proposes the Pi-GPS framework, which utilizes image information to eliminate textual ambiguities in geometry problems, thereby improving the accuracy of geometric problem-solving.
π― What it does: We designed and implemented PixelStitch, an unsupervised image stitching framework based on bidirectional pixel-level distortion, capable of achieving high-quality stitching while maintaining structural integrity.
PixTalk: Controlling Photorealistic Image Processing and Editing with Language
Marcos V. Conde (University of Wurzburg), Radu Timofte (University of Wurzburg)
CodeImage TranslationRestorationPrompt EngineeringMixture of ExpertsImageText
π― What it does: This paper presents PixTalk, a multi-task image processing model based on natural language prompts, capable of performing over 40 photography post-processing transformations at a speed of seconds on consumer GPUs for 12MP images.
PLA: Prompt Learning Attack against Text-to-Image Generative Models
Xinqi Lyu (Hong Kong Polytechnic University), Bin Xiao (Hong Kong Polytechnic University)
CodeGenerationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageTextMultimodality
π― What it does: This paper proposes a gradient-driven prompt learning attack framework (PLA) for black-box text-to-image models, which generates NSFW generation prompts that can bypass prompt filters and post-hoc safety checkers by utilizing sensitive knowledge encoding and multimodal similarity loss.
π― What it does: During the inference phase of the diffusion model, the introduction of sparse attention (Ξ±βEntmax) and a weighted compensation for dense attention enhance text alignment and image quality without the need for additional training or network evaluation.
π― What it does: The research objective is to enhance the spatial consistency and accuracy of CLIP in open vocabulary semantic segmentation through an untrained adaptive mechanism.
π― What it does: A clustering-based self-supervised point cloud masking modeling method called PointGAC is proposed, which utilizes an online codebook-guided teacher-student framework to align masked region features through clustering, thereby learning more generalized representations.
π― What it does: A unified dynamic 3D reconstruction framework called POMATO has been constructed, capable of simultaneously performing geometric estimation, point cloud matching, and temporal motion modeling, achieving end-to-end 3D reconstruction and point tracking from single frames to videos.
π― What it does: Proposes the Pose-Star framework, which can dynamically generate fine-grained human masks based on user-specified anatomical prompts (e.g., 'abdominal length') for open-world fashion image editing;
Power of Cooperative Supervision: Multiple Teachers Framework for Advanced 3D Semi-Supervised Object Detection
Jin-Hee Lee (DGIST), Kwon Soon (DGIST)
CodeObject DetectionAutonomous DrivingPoint Cloud
π― What it does: A multiple teacher framework (MultipleTeachers) is proposed for 3D semi-supervised object detection, accompanied by a pseudo-point generator (PointGen) for generating sparse LiDAR point clouds.
PRE-Mamba: A 4D State Space Model for Ultra-High-Frequent Event Camera Deraining
Ciyu Ruan (Shenzhen International Graduate School Tsinghua University), Xinlei Chen (Carnegie Mellon University)
CodeRestorationPoint Cloud
π― What it does: This paper proposes a rain algorithm PRE-Mamba based on event cameras, which implements point-level rain denoising using 4D event clouds representation, a spatiotemporal decoupling fusion module (STDF), and a multi-scale state space model (MS3M).
Jingwei Liu (Peking University), Mengdi Wang (Princeton University)
CodeLarge Language ModelAgentic AIVideoTextMultimodalityChain-of-Thought
π― What it does: An end-to-end paper-to-video summarization system called Preacher has been developed, which can automatically convert academic papers into structured and visual video summaries.
PriOr-Flow: Enhancing Primitive Panoramic Optical Flow with Orthogonal View
Longliang Liu (Huazhong University of Science and Technology), Xin Yang (Optics Valley Laboratory)
CodeOptical FlowImageVideo
π― What it does: A dual-branch framework called PriOr-Flow is proposed, which enhances panoramic optical flow estimation in polar regions using low-distortion priors from orthogonal perspectives.
π― What it does: The PRO-VPT framework is proposed, which dynamically adjusts the distribution of visual prompts through an iterative Prompt Relocation (PR) mechanism, achieving efficient parameterized fine-tuning of pre-trained visual models.
Probabilistic Inertial Poser (ProbIP): Uncertainty-aware Human Motion Modeling from Sparse Inertial Sensors
Min Kim (Korea Advanced Institute of Science and Technology), Sungho Jo (Korea Advanced Institute of Science and Technology)
CodePose EstimationGenerative Adversarial NetworkTime Series
π― What it does: Proposes the ProbIP model, which utilizes sparse IMU sensors to reconstruct full-body posture in a probabilistic manner, achieving real-time motion prediction without physical constraints.
Probabilistic Prototype Calibration of Vision-language Models for Generalized Few-shot Semantic Segmentation
Jie Liu (University of Amsterdam), Efstratios Gavves
CodeSegmentationTransformerVision Language ModelContrastive LearningImageMultimodality
π― What it does: A FewCLIP framework based on probabilistic prototype calibration is proposed for generalizing few-shot semantic segmentation, which can better capture intra-class variations and enhance generalization ability when sample scarcity occurs in new categories.
π― What it does: Proposes ProbMED, a multimodal visual-language pre-training framework that aligns lung X-rays, electrocardiograms, echocardiograms, and clinical texts through probabilistic alignment.
π― What it does: A three-stage evolutionary framework for prompt-based stability and plasticity (PHP) is proposed to address the issues of catastrophic forgetting and task interference in audio-visual multi-task incremental learning. The framework introduces a task-shared modality aggregation adapter (TMA) at the shallow level, uses a task-specific but modality-shared dynamic generation adapter (TMDG) at the middle level, and adds task-specific and modality-independent prompts (TMI) at the deep level, thereby achieving a layered knowledge transfer from commonality to personalization.
π― What it does: This paper proposes a replay-free online continual learning framework called PROL, which utilizes a single lightweight prompt generator and learnable scale-offset parameters to achieve stable-plastic learning on pre-trained models.
π― What it does: A HOT (Human-Object Contact) detection framework called P3HOT is proposed, which combines text prompts and human proximal perception to significantly enhance the localization and classification of contact areas using depth information and textual guidance.
Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM
Yatai Ji (University of Hong Kong), Ping Luo (University of Hong Kong)
CodeGenerationData SynthesisOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringDiffusion modelVideoText
π― What it does: This paper proposes and implements Prompt-A-Video, an automatic video prompt optimization system based on large language models, capable of generating high-quality video prompts that align with the preferences of video generation models.
Prompt-driven Transferable Adversarial Attack on Person Re-Identification with Attribute-aware Textual Inversion
Yuan Bian (Hunan University), Yaonan Wang (Hunan University)
CodeRecognitionAdversarial AttackTransformerVision Language ModelContrastive LearningImage
π― What it does: This paper proposes a transferable adversarial attack based on attribute-aware prompts (AP-Attack). By leveraging the image-text alignment capability of the visual-language model (CLIP), pedestrian images are mapped to attribute-level pseudo-words. These attribute semantics are utilized in the adversarial generator for fine-grained semantic disruption, resulting in adversarial samples with a high attack success rate across models and datasets.
π― What it does: An end-to-end proposal-driven visual localization framework, PropVG, is proposed, which integrates foreground proposal generation, Contrastive Reference Scoring (CRS), and Multi-Granularity Target Discrimination (MTD), achieving visual localization and segmentation without the need for a pre-trained detector.
π― What it does: The PAPN method is proposed, which improves self-supervised fine-grained visual representation through stage-wise evolution enhancement and prototype contrastive learning.
π― What it does: A Transformer-based multi-person extreme interaction motion prediction framework called PGformer is proposed, which can predict future 3D poses from past pose sequences.
Pruning All-Rounder: Rethinking and Improving Inference Efficiency for Large Vision Language Models
Wei Suo (Northwestern Polytechnical University), Yanning Zhang (Northwestern Polytechnical University)
CodeOptimizationComputational EfficiencyTransformerVision Language ModelMultimodality
π― What it does: A framework named Pruning All-Rounder (PAR) is proposed, which can significantly reduce inference FLOPs by jointly pruning visual tokens and model layers through a self-supervised preference optimization method without retraining the weights of large visual language models.
PRVQL: Progressive Knowledge-guided Refinement for Robust Egocentric Visual Query Localization
Bing Fan (University of North Texas), Heng Fan (University of Texas at Dallas)
CodeObject DetectionRetrievalTransformerVideo
π― What it does: A novel evolutionary knowledge-guided refinement framework named PRVQL is proposed to enhance the accuracy of locating visual query objects from first-person perspective videos.
π― What it does: This paper proposes the Pseudo-SD framework, which utilizes pseudo-labels to generate high-quality and diverse synthetic semantic segmentation data through Stable Diffusion for semi-supervised and cross-domain semantic segmentation tasks.
π― What it does: This paper studies the motion transferability of action recognition models in different contexts, proposing three benchmark datasets (Syn-TA, Kinetics400-TA, SomethingβSomethingβv2βTA) and systematically evaluating 13 state-of-the-art models.
π― What it does: This paper proposes a token purging-based method for test-time adaptation (TTA) in point cloud classification, called Purge-Gate (PG), which enhances robustness by removing the tokens most affected by domain shift before the attention layer.
Quantifying and Narrowing the Unknown: Interactive Text-to-Video Retrieval via Uncertainty Minimization
Bingqing Zhang (University of Queensland), Sen Wang (University of Queensland)
CodeRetrievalTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoText
π― What it does: An interactive text-to-video retrieval framework UMIVR is proposed, which explicitly quantifies and minimizes three types of uncertainties: text ambiguity, mapping uncertainty, and frame quality uncertainty, and iteratively improves queries through adaptive clarification questions.
π― What it does: A three-modal dataset R-LiViT of LiDAR, RGB, and thermal imaging for roadside intersections has been constructed, with a focus on annotating VRUs.
π― What it does: A semi-supervised thymus ultrasound image segmentation model RA-BUSSeg is proposed, combining two main modules: Adjacent Pixel Relationship Propagation (ARP) and Cross-layer Relationship Alignment (CRA) to enhance segmentation quality by utilizing pixel-level relationships.
Pedro R.A.S. Bassi, Zongwei Zhou (Johns Hopkins University)
CodeSegmentationGenerationConvolutional Neural NetworkLarge Language ModelImageTextMultimodalityBiomedical DataComputed TomographyBenchmark
π― What it does: This study presents AbdomenAtlas 3.0, a publicly available large-scale abdominal CT dataset containing 9,262 3D CT scans, tumor annotations for each voxel, and corresponding radiology reports. Additionally, the RadGPT framework was developed to convert tumor segmentation results into structured and narrative reports, enabling automated report generation through LLM. Benchmark evaluations of six CT report generation models on this dataset demonstrate that the segmentation-based RadGPT significantly outperforms end-to-end models in tumor detection sensitivity and specificity.
π― What it does: This paper proposes the RAGD (Regional-Aware Diffusion Model), which splits the original prompt into regional prompts. It first constructs each region using Regional Hard Binding in the early steps, and then integrates regional information using Regional Soft Refinement in the later steps, achieving precise control over location, attributes, and relationships, while supporting regional repainting of generated images without introducing additional patching models.
RARE: Refine Any Registration of Pairwise Point Clouds via Zero-Shot Learning
Chengyu Zheng (Nanjing University of Aeronautics and Astronautics), Mingqiang Wei (Nanjing University of Aeronautics and Astronautics)
CodeOptimizationDiffusion modelPoint Cloud
π― What it does: A method for zero-shot refinement of point cloud registration is proposed, utilizing a pre-trained diffusion model to extract diffusion features from depth maps, which are fused with existing geometric features to improve registration accuracy.
π― What it does: RealGeneral unifies multimodal image generation tasks as the next frame prediction task in video models, utilizing a pre-trained video diffusion model to generate various image conditional scenes in one go;