IEEE/CVF International Conference on Computer Vision · 743 papers
PRANC: Pseudo RAndom Networks for Compacting Deep Models
Parsa Nooralinejad (University of California), Hamed Pirsiavash (University of California)
CodeClassificationCompressionConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: Reparameterizing deep models as a linear combination of several randomly initialized, frozen base networks only requires saving the random seed and combination coefficients to restore the model.
Pre-training Vision Transformers with Very Limited Synthesized Images
Ryo Nakamura (National Institute of Advanced Industrial Science and Technology), Nakamasa Inoue (National Institute of Advanced Industrial Science and Technology)
🎯 What it does: A scheme was developed to pre-train a visual Transformer with minimal synthetic images by generating a fractal database (OFDB) that contains only one image per category and employing data augmentation during the pre-training phase.
🎯 What it does: A framework for image forgery localization based on Non-Exclusive Contrastive Learning (NCL) is proposed, which does not rely on pre-trained data and directly trains deep networks from the raw dataset.
Pretrained Language Models as Visual Planners for Human Assistance
Dhruvesh Patel (Meta), Ruta Desai (Meta)
CodeSegmentationGenerationTransformerLarge Language ModelVision Language ModelVideoMultimodality
🎯 What it does: This paper proposes the Visual Planning for Assistance (VPA) task and constructs the VLaMP model based on a two-stage approach of video segmentation and prediction to generate the next action sequence under the conditions of user-defined goals and video progress.
Privacy-Preserving Face Recognition Using Random Frequency Components
Yuxi Mi (Fudan University), Shuigeng Zhou (Fudan University)
CodeRecognitionSafty and PrivacyConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: This paper studies a privacy-preserving facial recognition method called PartialFace, which is achieved by randomly selecting high-frequency components in the frequency domain.
Probabilistic Triangulation for Uncalibrated Multi-View 3D Human Pose Estimation
Boyuan Jiang (Institute of Computing Technology, Chinese Academy of Sciences), Shihong Xia (Institute of Computing Technology, Chinese Academy of Sciences)
CodePose EstimationImageVideo
🎯 What it does: This paper proposes a Probabilistic Triangulation module that can achieve 3D human pose estimation in uncalibrated multi-view scenarios.
🎯 What it does: This paper proposes a text-video retrieval framework called ProST based on advanced spatial-temporal prototype matching. It first generates spatial prototypes to match local objects and phrases, and then generates temporal prototypes to match events and sentences, achieving multi-granularity alignment.
Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval
Chaorui Deng (Australia Institute of Machine Learning, University of Adelaide), Qi Wu (Australia Institute of Machine Learning, University of Adelaide)
🎯 What it does: In text-video retrieval, global semantic modeling of videos is achieved using the image encoder of CLIP through a switchable three-dimensional Prompt Cube, and fine-grained semantics are enhanced with auxiliary video subtitle objectives.
🎯 What it does: This paper proposes a text-driven image editing method based on diffusion models, utilizing Prompt Tuning Inversion to encode the original image information into a learnable conditional embedding, which is then linearly interpolated with the target text embedding to achieve image editing.
PromptCap: Prompt-Guided Image Captioning for VQA with GPT-3
Yushi Hu (University of Washington), Jiebo Luo (University of Rochester)
CodeGenerationRetrievalTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes PROMPTCAP, a visual description model controlled by natural language prompts, which transforms images into customized textual descriptions tailored to questions, enabling black-box large language models like GPT-3 to understand images and perform knowledge-driven visual question answering.
Prototypes-oriented Transductive Few-shot Learning with Conditional Transport
Long Tian (Xidian University), Bo Chen (Xidian University)
CodeClassificationContrastive LearningImage
🎯 What it does: This study investigates transductive few-shot learning with class imbalance, proposing the PUTM model that achieves unbiased statistical transfer through Conditional Transport (CT), improving prototype generation and classification.
🎯 What it does: This paper proposes RandBox, an open-world object detection framework that uses randomly generated boxes during training, capable of recognizing known categories while labeling unannotated objects as 'unknown'.
Random Sub-Samples Generation for Self-Supervised Real Image Denoising
Yizhong Pan (Sichuan University), Chao Ren (Sichuan University)
CodeRestorationImageBenchmark
🎯 What it does: The SDAP framework is proposed, achieving unsupervised real image denoising through Random Subsample Generation (RSG) and Circular Sampling Differential Loss (CSDBSN).
🎯 What it does: This paper proposes a HDR reconstruction framework called RawHDR based on a single raw image, which can directly generate a 20-bit HDR image from 14-bit raw data.
Read-only Prompt Optimization for Vision-Language Few-shot Learning
Dongjun Lee (Korea University), Hyunwoo J. Kim (Korea University)
CodeClassificationDomain AdaptationOptimizationTransformerPrompt EngineeringVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: This paper proposes a Read-only Prompt Optimization (RPO) method, which prevents unnecessary shifts in the internal representations of pre-trained vision-language models during fine-tuning through a masked attention mechanism and special token initialization, thereby achieving efficient and robust few-shot learning.
🎯 What it does: Proposed the 4D Scene Context Graph Generation (CGG) task and constructed the first multi-view RGB video dataset RealGraph, while providing the baseline model MCGNet;
🎯 What it does: This paper proposes a multi-task learning framework called ROG, which combines object-level classification and global ranking tasks to enhance long-tail detection performance.
🎯 What it does: A novel Reconfigurable Convolution (RC) module has been developed, decoupling channel and spatial computations, and achieving n×n convolution through n² 1D LUTs, significantly enhancing the receptive field and reducing storage in single image super-resolution.
🎯 What it does: A recursive video lane detection algorithm RVLD has been developed, which utilizes single-frame historical information to achieve continuous video lane detection through motion estimation and feature reconstruction.
Jing Zhao (East China Normal University), Qingli Li (East China Normal University)
CodeObject DetectionTransformerImage
🎯 What it does: A recursive decoder is proposed, utilizing parameter sharing and bounding box position encoding, significantly improving the performance of end-to-end region detectors while reducing the model parameter count.
🎯 What it does: The RED-PSM method is proposed, combining low-rank partially separable models and denoising-based regularization to address the undersampling reconstruction problem in dynamic imaging.
🎯 What it does: A weakly supervised image segmentation framework is proposed that uses only text expressions as supervision, optimizing the positioning of targets through text-image response and generating pseudo-labels to train the segmentation network.
🎯 What it does: An end-to-end RegFormer network is proposed for large-scale point cloud registration, eliminating the dependence on keypoint detection, feature description, and RANSAC post-processing, and capable of directly estimating rigid transformations from raw LiDAR point clouds.
🎯 What it does: This study investigates a domain continual learning (DCL) facial spoof detection (FAS) model under conditions of no replay buffer and low sample size, and proposes a new replay-free training framework.
🎯 What it does: By precomputing and storing the outputs of a strong teacher model under various data augmentations on the training set, a reinforced dataset (such as ImageNet+) is constructed to improve the accuracy and robustness of any model without additional training costs.
Relightify: Relightable 3D Faces from a Single Image via Diffusion Models
Foivos Paraperas Papantoniou (Imperial College London), Stefanos Zafeiriou (Imperial College London)
CodeRestorationGenerationDiffusion modelImage
🎯 What it does: Using a single facial image, this paper combines an unsupervised diffusion model to simultaneously recover UV textures and BRDF (diffuse reflection, specular reflection, normals), generating a 3D facial model that can be rendered under arbitrary lighting.
RenderIH: A Large-Scale Synthetic Dataset for 3D Interacting Hand Pose Estimation
Lijun Li (Alibaba Group), Chen Chen (University of Central Florida)
CodeData SynthesisPose EstimationTransformerImage
🎯 What it does: This paper proposes a large-scale synthetic dataset RenderIH to enhance 3D hand pose estimation under single RGB images, and conducts experimental validation based on the Transformer-based TransHand network.
🎯 What it does: Proposes a representation difference-aware distillation (RDD) method based on information bottleneck to enhance the performance of ultra-small 3D detectors.
🎯 What it does: A Residual Pattern Learning (RPL) module and Context Robust Contrastive Learning (CoroCL) have been designed to achieve pixel-level Out-of-Distribution (OoD) detection on a frozen semantic segmentation network while maintaining segmentation accuracy.
🎯 What it does: Proposes the EoRaS method, which utilizes supervised visible masks and multi-view information for video intangible segmentation, achieving joint reasoning of shape and viewpoint priors through a BEV translation network and a multi-view fusion layer.
Rethinking Data Distillation: Do Not Overlook Calibration
Dongyao Zhu (University of California San Diego), Dongkuan Xu (North Carolina State University)
CodeKnowledge DistillationImage
🎯 What it does: This study investigates the calibration problem of networks trained through data distillation (DDNN) and proposes two methods, Masked Temperature Scaling (MTS) and Masked Distillation Training (MDT), to address it.
🎯 What it does: A unified lightweight basic block called Meta Mobile Block (MMB) is proposed, from which an improved Inverted Residual Mobile Block (iRMB) is derived. Based on this, an efficient model EMO, consisting solely of iRMB, is constructed for dense prediction tasks such as image classification, object detection, and semantic segmentation.
Rethinking Point Cloud Registration as Masking and Reconstruction
Guangyan Chen (Beijing Institute of Technology), Yufeng Yue (Beijing Institute of Technology)
CodeObject DetectionTransformerPoint Cloud
🎯 What it does: View point cloud registration as a masking and reconstruction task, proposing the Mask Reconstruction Auxiliary Network (MRA) to assist the main network in learning fine-grained geometry and overall structure, and based on this, designing the Mask Reconstruction Transformer (MRT) to achieve efficient and accurate registration.
🎯 What it does: A two-stage pose estimation framework that combines low-level detection with conditional top-level processing (BUCTD) is proposed, utilizing a low-level pose detector to generate pose hints as conditional inputs for the top network.
🎯 What it does: This paper proposes a lightweight visual Transformer architecture named EfficientFormerV2, designed for efficient inference on mobile devices.
Revisit PCA-based Technique for Out-of-Distribution Detection
Xiaoyuan Guan (Sun Yat-sen University), Ruixuan Wang (Sun Yat-sen University)
CodeAnomaly DetectionAuto EncoderImage
🎯 What it does: A post-processing method that integrates the regularized PCA reconstruction error with energy scores is proposed to improve OOD detection in deep learning models.
🎯 What it does: This paper proposes an unsupervised domain adaptation framework for 3D object detection called ReDB, which utilizes reliable, diverse, and class-balanced pseudo-labels to achieve multi-class self-training.
RFD-ECNet: Extreme Underwater Image Compression with Reference to Feature Dictionary
Mengyao Li (Shanghai University), Zheyin Wang (Shanghai University)
CodeCompressionAuto EncoderImage
🎯 What it does: A limit underwater image compression network RFD-ECNet based on an underwater multi-scale feature dictionary is designed, which can remove redundancy between different underwater images through a reference dictionary.
🎯 What it does: This paper proposes a physical adversarial attack based on reflected light (RFLA), which generates adjustable geometric shapes and colors of reflected light to deceive DNN models by using specular reflection of sunlight or flashlights, combined with colored transparent plastic sheets and paper cutouts.
🎯 What it does: The RICO method is proposed, achieving object-level decomposition and reconstruction by regularizing the invisible regions in indoor scenes.
RLIPv2: Fast Scaling of Relational Language-Image Pre-Training
Hangjie Yuan (Zhejiang University), Deli Zhao (Alibaba Group)
CodeRecognitionObject DetectionTransformerVision Language ModelImageTextMultimodality
🎯 What it does: The RLIPv2 model is proposed, which combines the rapidly converging Asymmetric Language-Image Fusion (ALIF) and large-scale pseudo-labeled scene graph data to achieve large-scale relational language-image pre-training.
🎯 What it does: This work proposes RLSAC, a reinforcement learning-based sampling consensus framework that achieves end-to-end robust model estimation.
🎯 What it does: Established the Robo3D benchmark, defined eight types of real-world LiDAR noise and faults, and systematically evaluated the robustness of 34 3D detection and segmentation models under different severity levels; also proposed two techniques, density-sensitive training framework and variable voxelization, to enhance robustness.
🎯 What it does: Evaluate the robustness of diffusion-based adversarial purification and propose more rigorous evaluation criteria and a multi-step purification strategy with gradual noise scheduling.
🎯 What it does: A robust training framework called AugHFL is proposed to address the issue of data corruption in heterogeneous federated learning, which can simultaneously suppress the negative effects of internal and external data corruption during both local training and global collaborative learning phases.
🎯 What it does: This paper proposes Token-aware Average Pooling (TAP) and Attention Diversification Loss (ADL), which alleviate the token overfocusing problem and enhance robustness by allowing ViT to focus more on local neighborhoods during the self-attention process and suppress the attention similarity between different tokens.
🎯 What it does: A white-box image beautification framework RSFNet is proposed, utilizing parallel region-specific filters to achieve fine-grained color adjustments and providing editable filter parameters.
🎯 What it does: A post-hoc OOD detection method called SAFE is proposed, which utilizes sensitive feature vectors extracted from the residual convolution + BatchNorm layers of a pre-trained detector backbone, and trains an auxiliary MLP to distinguish between ID and OOD detection results.
🎯 What it does: A spectral domain-based geometric adversarial attack method (SAGA) is proposed for 3D mesh autoencoders, enabling the model to output a geometric shape nearly identical to the target mesh after the input is subjected to minor perturbations.
🎯 What it does: This paper proposes AdaptPoint, an adaptive point cloud enhancement framework that improves the model's robustness to real-world corruption by generating local deformations and occlusions based on the point cloud structure.
SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data
Mohammad Zohaib (Italian Institute of Technology), Alessio Del Bue (Italian Institute of Technology)
CodeObject DetectionPose EstimationPoint Cloud
🎯 What it does: This paper proposes a completely unsupervised 3D keypoint detection method called SC3K, which can efficiently and robustly infer semantically consistent keypoints that closely adhere to surfaces in point cloud data under arbitrary rotations, noise, and downsampling.
🎯 What it does: A scalable multi-temporal remote sensing change data generation method called Changen is proposed based on the Generative Probability Change Model (GPCM), which can automatically generate controllable change pairs from single-time images and their semantic segmentation images.
🎯 What it does: A new visual Transformer backbone network called Scale-Aware Modulation Transformer (SMT) is designed, which combines convolution with Transformer to achieve multi-scale feature fusion and local-to-global dependency modeling.
Scaling Data Generation in Vision-and-Language Navigation
Zun Wang (Australian National University), Yu Qiao (OpenGVLab Shanghai AI Laboratory)
CodeGenerationData SynthesisGraph Neural NetworkVision Language ModelGenerative Adversarial NetworkImageTextMultimodality
🎯 What it does: This paper proposes the ScaleVLN large-scale visual and language navigation data generation paradigm, utilizing over 1200 3D scenes from HM3D and Gibson to construct a fully covered, obstacle-free navigation map, restore rendered images, and generate 4.9 million R2R-style instruction-trajectory pairs.
Wenwen Tong (Shanghai AI Laboratory), Hongyang Li (Shanghai AI Laboratory)
CodeObject DetectionSegmentationAutonomous DrivingTransformerSimultaneous Localization and MappingOptical FlowPoint CloudBenchmark
🎯 What it does: Proposes the OccNet framework and the OpenOcc benchmark, utilizing multi-view vision to construct dense 3D occupancy maps, achieving occupancy prediction and supporting multiple tasks.
🎯 What it does: This paper studies a general text understanding pre-training method called SCOB, which bridges the domain gap between document images and scene text images through character-level supervised contrastive learning and online text rendering.
🎯 What it does: This paper proposes a category-aware group self-support learning framework (GSS) to enhance feature extraction and mutual distillation effects for multimodal brain tumor segmentation under missing modality conditions.
🎯 What it does: To address the issue of sliding window scale limitations in high-resolution remote sensing image segmentation, this paper proposes GeoAgent—a scale-adaptive semantic segmentation framework based on reinforcement learning, which can dynamically select appropriate image block scales to obtain richer contextual information, thereby improving segmentation accuracy.
🎯 What it does: SegGPT proposes a universal model capable of completing various segmentation tasks in a single context reasoning. By employing a randomly colored in-context learning framework during training on different segmentation data, the model relies on context rather than specific colors to accomplish tasks.
🎯 What it does: Proposes the SegPrompt mechanism and the LVIS-OW benchmark to achieve better evaluation and improvement of open-world instance segmentation.
🎯 What it does: A storage-efficient visual training framework called SeiT is proposed, which uses 1% pixel storage to compress images into 1024 discrete tokens and directly trains ViT.
Self-Evolved Dynamic Expansion Model for Task-Free Continual Learning
Fei Ye (University of York), Adrian G. Bors (University of York)
CodeOptimizationMixture of ExpertsAuto EncoderImage
🎯 What it does: Proposed and implemented a Self-Evolving Dynamic Expansion Model (SEDEM) for task-agnostic continual learning, capable of automatically determining whether to add new experts and conducting incremental training in data streams.
🎯 What it does: Proposed the PromptSRC framework, which utilizes self-regularization to learn prompts on CLIP, balancing task-specific and general features.
🎯 What it does: A Self-similarity driven Scale-invariant Learning (SSL) framework is proposed to address the scale variation problem in weakly supervised person retrieval.
🎯 What it does: A self-supervised character-level distillation framework (CCD) is proposed, which treats each character as a basic learning unit for representation learning of text images through self-supervised character segmentation and geometric transformation alignment.
🎯 What it does: This paper proposes a self-supervised cross-view representation reconstruction network called SCORER, which aims to learn robust differential representations and generate differential explanations in the presence of view pseudo-transformations.
🎯 What it does: To address the direction sensitivity and environmental dependence in self-supervised monocular depth estimation, a Direction-Aware Cumulative Convolutional Network (DaCCN) is proposed.
🎯 What it does: In the pre-training phase, the SemCL method is proposed, which generates contrastive samples of objects and their surrounding environments by utilizing publicly available semantic annotations to construct a new contrastive learning pretext task;
Semantic-Aware Implicit Template Learning via Part Deformation Consistency
Sihyeon Kim (Korea University), Hyunwoo J. Kim (Korea University)
CodeSegmentationGenerationAuto EncoderPoint Cloud
🎯 What it does: Learning a semantic-aware implicit template network that achieves unified representation and high-quality correspondence for different shapes through self-supervised segmentation feature-guided templates and deformation fields;
Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos
Rui Qian (Chinese University of Hong Kong), Dahua Lin (Chinese University of Hong Kong)
CodeObject DetectionSegmentationTransformerVideo
🎯 What it does: A self-supervised framework SMTC is proposed, which combines high-level semantics and low-level temporal correspondence to achieve unsupervised video object learning, utilizing semantic-aware mask slot attention for semantic decomposition and instance recognition.
🎯 What it does: This paper proposes a method that utilizes Semantic Consistent Feature Search (SCFS) in self-supervised contrastive learning to adaptively find semantically consistent feature regions for comparison, thereby alleviating the semantic inconsistency issues caused by data augmentation and enhancing the model's focus on target regions.
CodeAutonomous DrivingAdversarial AttackGraph Neural NetworkAuto EncoderGenerative Adversarial NetworkTime Series
🎯 What it does: A semi-supervised semantic-guided adversarial training method has been developed to enhance the robustness and generalization ability of trajectory prediction models under adversarial attacks.
Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models
Dong Lu (Southern University of Science and Technology), Feng Zheng (Monash University)
CodeRetrievalAdversarial AttackTransformerVision Language ModelMultimodality
🎯 What it does: This study investigates the adversarial transferability of visual-language pre-trained models and proposes the Set-level Guidance Attack (SGA) method.
🎯 What it does: A self-guided Transformer (SG-Former) is designed to dynamically reallocate tokens based on the importance map predicted by the model itself, achieving efficient fine-grained global self-attention modeling through a mixed scale attention mechanism that integrates both local and global attention within the same layer.
🎯 What it does: This paper proposes a shape anchor point guided learning strategy called AncLearn, and integrates it into the AncRec framework to achieve unified indoor scene understanding from detection to reconstruction.
🎯 What it does: An online data augmentation method based on edge deformation (SDbOA) is proposed, which diversifies the shape of objects through TPS deformation, thereby reducing the texture bias of CNNs and enhancing the model's reliance on shape features.
🎯 What it does: This paper proposes a semi-supervised learning framework called ShrinkMatch, which reduces the category space by automatically removing categories that are confused with the highest predicted category, allowing uncertain samples to gain sufficient confidence in the new space and be utilized;
🎯 What it does: A simple and scalable self-supervised learning framework SiLK has been designed and implemented for learning image keypoint detection and description.
Simple Baselines for Interactive Video Retrieval with Questions and Answers
Kaiqu Liang (Princeton University), Samuel Albanie (University of Cambridge)
CodeRetrievalTransformerLarge Language ModelVision Language ModelVideoText
🎯 What it does: This paper proposes an interactive video retrieval framework based on question answering, utilizing a video question answering model to simulate user responses for interactive retrieval.
SimpleClick: Interactive Image Segmentation with Simple Vision Transformers
Qin Liu (University of North Carolina at Chapel Hill), Marc Niethammer (University of North Carolina at Chapel Hill)
CodeSegmentationTransformerImageBiomedical Data
🎯 What it does: An interactive image segmentation method called SimpleClick is proposed, which is based on a standard ViT backbone and achieves efficient segmentation through symmetric embedding of click information and a simple feature pyramid.
Single Image Reflection Separation via Component Synergy
Qiming Hu (Tianjin University), Xiaojie Guo (Tianjin University)
CodeRestorationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a new single-image reflection separation model that utilizes a learnable residual term and a dual-stream semantic-aware network to achieve more complete separation of reflection and transmission layers.
🎯 What it does: Proposes SSDNeRF, a unified framework for single-stage training that can simultaneously perform unconditional 3D generation and sparse view reconstruction;
🎯 What it does: A high-resolution virtual try-on network based on garment structure, COTTON, is proposed, which can achieve precise deformation through garment key points and segmentation, and supports try-on for different sizes.
Social Diffusion: Long-term Multiple Human Motion Anticipation
Julian Tanke (University of Bonn), Cem Keskin (Reality Labs Research)
CodeGenerationPose EstimationDiffusion modelVideo
🎯 What it does: A multi-person human action prediction framework based on diffusion models—Social Diffusion—has been proposed, which generates motion sequences feasible for social interaction while maintaining the authenticity of individual postures.
🎯 What it does: Learn contact surfaces through source-free deep networks and the 'pop-out' prior, converting depth to semantics to achieve cross-domain and cross-task object segmentation.
🎯 What it does: This paper proposes a source-free domain adaptive human pose estimation task and designs an end-to-end framework consisting of a source model, a transition model, and a target model in a three-layer structure;
🎯 What it does: Proposes the SpaceEvo method, which automatically constructs a hardware-specific INT8 quantization-friendly search space and trains a quantization-for-all super network within that space, ultimately resulting in the SEQnet series of efficient models.
🎯 What it does: Using sparse candidate boxes and sparse representation, we construct a 3D object detection framework called SparseFusion for LiDAR and camera sensors. Instance-level features are extracted through parallel single-modal detectors, and then the camera candidate boxes are projected into the LiDAR coordinate system. A lightweight self-attention module is used to fuse the two modal features in a unified 3D space, ultimately yielding high-quality 3D frames.