IEEE/CVF Conference on Computer Vision and Pattern Recognition Β· 830 papers
CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution
Jiezhang Cao (ETH Zurich), Luc Van Gool (ETH Zurich)
CodeRestorationSuper ResolutionImage
π― What it does: A continuous implicit attention network called CiaoSR is proposed for arbitrary scale image super-resolution, which can be seamlessly integrated with any SR backbone network.
π― What it does: This paper proposes a sign language retrieval framework CiCo based on cross-lingual contrastive learning, which can achieve natural language query retrieval of corresponding sign language videos, as well as sign language video retrieval of corresponding text.
π― What it does: This paper proposes a Category Adaptive Label Smoothing (CALS) method based on an improved Augmented Lagrangian Multiplier (ALM) algorithm to simultaneously optimize accuracy and calibration in deep network training.
Class Relationship Embedded Learning for Source-Free Unsupervised Domain Adaptation
Yixin Zhang (University of Science and Technology of China), Weinan He (University of Science and Technology of China)
CodeDomain AdaptationContrastive LearningImage
π― What it does: In the unsupervised domain adaptation scenario with no data in the source domain, knowledge transfer is achieved by utilizing the class relationships of the pre-trained source model and implementing contrastive learning through class relationship embedding;
Class-Incremental Exemplar Compression for Class-Incremental Learning
Zilin Luo (Singapore Management University), Qianru Sun (Singapore Management University)
CodeCompressionImage
π― What it does: An adaptive mask compression method CIM is proposed, which utilizes CAM and learnable activation functions to compress samples of old classes in class-incremental learning;
π― What it does: A framework called CLIP-ES is proposed, which utilizes a frozen CLIP model and a text-driven strategy to generate pseudo-masks and train segmentation models for weakly supervised semantic segmentation without training additional networks.
CLIP the Gap: A Single Domain Generalization Approach for Object Detection
Vidit Vidit (EPFL), Mathieu Salzmann (EPFL)
CodeObject DetectionDomain AdaptationAutonomous DrivingPrompt EngineeringVision Language ModelImage
π― What it does: Utilize the pre-trained visual language model CLIP for semantic enhancement to train a single-domain generalization object detector.
CLIP2Protect: Protecting Facial Privacy Using Text-Guided Makeup via Adversarial Latent Search
Fahad Shamshad (Mohamed Bin Zayed University of AI), Karthik Nandakumar (Mohamed Bin Zayed University of AI)
CodeGenerationSafty and PrivacyAdversarial AttackGenerative Adversarial NetworkImage
π― What it does: This paper proposes a text-guided concealment method based on a pre-trained generative model (StyleGAN) to generate 'naturalized' facial images that can mislead black-box face recognition systems without affecting user experience.
π― What it does: This paper proposes the CLIP2Scene framework, which transfers the image-text pre-training knowledge of CLIP to 3D point cloud networks, achieving unsupervised 3D semantic segmentation and label-efficient fine-tuning.
CLIPPO: Image-and-Language Understanding From Pixels Only
Michael Tschannen (Google Research), Neil Houlsby (Google Research)
CodeRetrievalTransformerVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: A single-tower Vision Transformer (CLIPPO) is proposed and trained, which can handle image, text, and text-image mixed tasks simultaneously using only pixel input.
CLOTH4D: A Dataset for Clothed Human Reconstruction
Xingxing Zou (Hong Kong Polytechnic University), Waikeung Wong (Hong Kong Polytechnic University)
CodeMeshBenchmark
π― What it does: A large-scale high-quality 4D clothing dataset CLOTH4D is proposed, and this dataset is used to evaluate and improve existing garment-human reconstruction methods.
CNVid-3.5M: Build, Filter, and Pre-Train the Large-Scale Public Chinese Video-Text Dataset
Tian Gan (Shandong University), Qingpei Guo (Ant Group)
CodeRetrievalTransformerVision Language ModelVideoTextMultimodality
π― What it does: This paper constructs and publicly releases CNVid-3.5M, which contains 3.5 million Chinese video-text pairs, and conducts pixel-level pre-training and evaluation on this dataset.
π― What it does: A reinforcement learning-based co-speech gesture synthesis framework called RACER is proposed, which generates gesture sequences that are synchronized and coherent with speech.
π― What it does: A Collaborative Diffusion framework is proposed, which achieves multi-modal (text, mask, etc.) face generation and editing through a pre-trained unimodal diffusion model without the need for retraining.
π― What it does: This paper proposes a Collaborative Noisy Label Cleaner (CLC) framework that utilizes the noisy labels present in movie trailers to learn movie highlight detection.
Collaborative Static and Dynamic Vision-Language Streams for Spatio-Temporal Video Grounding
Zihang Lin (Sun Yat-sen University), Wei-Shi Zheng (Tencent)
CodeRecognitionObject DetectionTransformerVision Language ModelVideoText
π― What it does: This paper proposes a two-stream visual-language framework for the task of Spatio-Temporal Video Grounding, where the static stream focuses on the appearance of objects within frames, and the dynamic stream focuses on motion information across frames. A cross-stream collaboration module is implemented to achieve complementary information transfer between the two streams, thereby locating the spatial bounding box and temporal segment of the target object.
π― What it does: The CoMFormer model is proposed, which can continuously learn and simultaneously handle semantic segmentation and panoptic segmentation tasks without retraining.
Command-Driven Articulated Object Understanding and Manipulation
Ruihang Chu (Chinese University of Hong Kong), Jiaya Jia (Chinese University of Hong Kong)
CodeRobotic IntelligencePoint Cloud
π― What it does: This paper proposes a user-instruction-based joint manipulation framework called Cart, which can identify and manipulate movable components with only a single point cloud, achieving predictions of joint types, axes, displacements/angles, and actual motion control.
π― What it does: A complete-part 4D knowledge distillation method is proposed, using a teacher-student framework to learn point cloud sequence representations.
π― What it does: A compressed sensing video super-resolution model CAVSR is proposed, which can adapt to input videos at different compression levels, achieving high-quality super-resolution of compressed videos.
Conditional Text Image Generation With Diffusion Models
Yuanzhi Zhu (Alibaba DAMO Academy), Cong Yao (Alibaba DAMO Academy)
CodeGenerationDiffusion modelImageText
π― What it does: A conditional text-image generation framework based on diffusion models, CTIG-DM, is proposed, which can control the generated text images through three conditions: image, text, and style.
Conjugate Product Graphs for Globally Optimal 2D-3D Shape Matching
Paul Roetzer (University of Bonn), Florian Bernard (University of Bonn)
CodeOptimizationGraph Neural NetworkMesh
π― What it does: This paper proposes a globally optimal 2D-3D shape matching method based on conjugate product graphs, achieving continuous and unbiased matching from 2D contours to 3D meshes by introducing higher-order costs in the graph to implement local rigidity constraints.
π― What it does: This paper proposes a single-stage Transformer model called RoomFormer, which can directly predict the polygons of all rooms, room types, and structural elements such as doors and windows from the top-down density maps obtained from 3D point cloud projections.
π― What it does: This paper proposes a unified pre-training framework for 3D point clouds and text, achieving fine-grained interaction between point cloud and language features through context-aware spatial semantic alignment and mutual masking modeling, and transferring the pre-trained model to multi-tasks such as 3D visual localization, dense description, and question answering.
π― What it does: A context-aware pre-training framework (CP) is proposed to achieve blind image decomposition tasks that remove various mixed noises in a one-time manner.
π― What it does: A progressive image compression algorithm CTC based on Trit-Plane coding is proposed, utilizing context models to achieve more efficient bitrate control and image reconstruction.
π― What it does: A pseudo-label correction framework based on continuous implicit neural representation (Continuous RMM) is proposed for unsupervised domain adaptation semantic segmentation, which can enhance segmentation performance through self-training in the absence of labels in the target domain.
π― What it does: This paper proposes the CorrNet network, which explicitly captures body trajectories in continuous sign language videos through correlation and recognition modules, thereby enhancing continuous sign language recognition performance.
π― What it does: A framework that integrates contrastive learning with mean teacher self-supervised learning is proposed for unsupervised domain adaptation in object detection.
Contrastive Semi-Supervised Learning for Underwater Image Restoration via Reliable Bank
Shirui Huang (Xidian University), Yunsong Li (Xidian University)
CodeRestorationContrastive LearningImage
π― What it does: Proposes the Semi-UIR semi-supervised underwater image restoration framework, utilizing unlabeled data to enhance model generalization capabilities.
π― What it does: This paper proposes the ConvNeXt V2 network architecture and designs a fully convolutional sparse Mask-AutoEncoder (FCMAE) for self-supervised pre-training, addressing the mismatch between traditional ConvNets and MAE; it achieves performance improvements on multiple tasks such as ImageNet, COCO, and ADE20K.
ConZIC: Controllable Zero-Shot Image Captioning by Sampling-Based Polishing
Zequn Zeng (Xidian University), Zhengjue Wang (Xidian University)
CodeGenerationTransformerVision Language ModelContrastive LearningImageText
π― What it does: A sampling-based non-autoregressive model, Gibbs-BERT, combined with CLIP, proposes a zero-shot image description framework called ConZIC, which can generate diverse and controllable image descriptions under unsupervised training conditions.
CORA: Adapting CLIP for Open-Vocabulary Detection With Region Prompting and Anchor Pre-Matching
Xiaoshi Wu (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)
CodeObject DetectionTransformerPrompt EngineeringVision Language ModelContrastive LearningImage
π― What it does: A CLIP-based open vocabulary word detection framework called CORA is proposed, which achieves open vocabulary detection without additional image-text data through region prompts and anchor pre-matching.
π― What it does: This paper proposes a novel Transformer-based semantic correspondence frameworkβACTR, which achieves end-to-end training of dense visual correspondence through asymmetric feature learning and matching flow super-resolution.
π― What it does: This paper proposes a new facial image quality assessment method CR-FIQA, which predicts image quality by utilizing the relative classifiability of samples during the training process.
CREPE: Can Vision-Language Foundation Models Reason Compositionally?
Zixian Ma (Stanford University), Ranjay Krishna (University of Washington)
CodeRetrievalTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
π― What it does: Designed and released the CREPE benchmark to evaluate the combinatorial reasoning capabilities of visual language models in terms of systematicity and productivity.
Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval
Ding Jiang (Wuhan University), Mang Ye (Wuhan University)
CodeRetrievalTransformerVision Language ModelContrastive LearningImageText
π― What it does: This paper proposes a cross-modal implicit relationship reasoning and alignment framework (IRRA), which utilizes the Masked Language Modeling (MLM) task to achieve fine-grained relationship learning between vision and text, and optimizes cross-modal matching through Similarity Distribution Matching loss.
Curricular Contrastive Regularization for Physics-Aware Single Image Dehazing
Yu Zheng (Ocean University of China), Yong Du (Ocean University of China)
CodeRestorationConvolutional Neural NetworkTransformerContrastive LearningImagePhysics Related
π― What it does: This study focuses on single image dehazing and proposes a curriculum contrastive regularization and physics-aware dual-branch network called C2PNet.
Curricular Object Manipulation in LiDAR-Based Object Detection
Ziyue Zhu (Nankai University), Jian Yang (Nankai University)
CodeObject DetectionAutonomous DrivingPoint Cloud
π― What it does: A framework that integrates curriculum learning into LiDAR 3D object detection is proposed, which includes two parts: loss reweighting and data augmentation.
π― What it does: Using features extracted by the self-supervised visual Transformer (DINO), multiple rough object masks are generated through MaskCut, and then a general object detection and instance segmentation model is trained using DropLoss and multiple rounds of self-training. The resulting model can perform zero-shot detection and segmentation across various visual domains without using any annotations.
π― What it does: A data augmentation method named CutMIB, which stands for 'Crop-Mix-Paste', is designed and implemented. It enhances the training effect of light field super-resolution networks by randomly cropping low-resolution patches from the same position in multi-view light fields, averaging and mixing them, and then pasting them onto the corresponding high-resolution views, all while keeping the network structure unchanged.
π― What it does: This paper proposes a single-stream continuous sign language recognition framework called CVT-SLR based on contrastive visual-text transformation, which integrates pre-trained visual models with variational autoencoders to incorporate pre-trained linguistic knowledge and enhances performance through contrastive cross-modal alignment.
π― What it does: This paper predicts a local grid subset near user-selected points by training a neural network, ensuring low distortion and large area during UV mapping, and enabling quick results in interaction.
π― What it does: This paper studies a diversified, aggregated, and repeated model weight strategy during training (DART) to enhance the generalization ability of neural networks both within and outside the domain.
Data-Free Knowledge Distillation via Feature Exchange and Activation Region Constraint
Shikang Yu (Institute of Computing Technology Chinese Academy of Sciences), Shuqiang Jiang (Institute of Computing Technology Chinese Academy of Sciences)
π― What it does: A data-free knowledge distillation method called SpaceshipNet is proposed, which is based on Channel-level Feature Exchange (CFE) and Multi-Scale Spatial Activation Region Consistency (mSARC) constraints. It enables high-quality student model training by generating diverse synthetic images without accessing the original data.
π― What it does: Construct a unified DATE framework to address product retrieval (PR) and localization (PG) tasks, and achieve unsupervised domain adaptation (PG-DA).
π― What it does: A decoupled multimodal distillation framework (DMD) is proposed, which divides multimodal features into shared (modal-independent) and exclusive (modal-specific) parts, using homogeneous graph distillation and heterogeneous graph distillation to enhance emotion recognition performance.
π― What it does: This paper proposes a semi-weakly supervised semantic segmentation method called Decoupled Semantic Prototypes (DSP) that can simultaneously utilize pixel-level annotations, box annotations, point annotations, image-level labels, and unlabeled images.
π― What it does: A dual-layer memory framework (BMKP) is proposed, which learns new tasks through working memory, stores knowledge in long-term memory, and compresses parameters in working memory into shared pattern bases using knowledge projection, which are then stored in long-term memory.
Deep Arbitrary-Scale Image Super-Resolution via Scale-Equivariance Pursuit
Xiaohang Wang (Shanghai Jiao Tong University), Yutian Liu (Shanghai Jiao Tong University)
CodeRestorationSuper ResolutionTransformerImage
π― What it does: A scale-invariant arbitrary scale image super-resolution framework EQSR is proposed, which can achieve high-quality super-resolution for any scaling factor using the same model.
π― What it does: A method for latent space editing based on a curvilinear coordinate system, DeCurvEd, is proposed, and CurvilinearGANSpace is implemented on GANs to achieve nonlinear and interchangeable semantic attribute editing.
π― What it does: This paper proposes DSTNet, a lightweight deep convolutional network for video deblurring, which is centered around a channel-gated dynamic network, a discriminative temporal feature fusion module, and a wavelet feature propagation mechanism, avoiding traditional optical flow or deformable convolution alignment methods.
π― What it does: This paper studies and improves the IoU loss in 3D object detection, proposing the Gradient-Corrected IoU (GCIoU) loss and a gradient scaling strategy to achieve faster and more accurate bounding box regression.
Chengkun Wang (Tsinghua University), Jiwen Lu (Tsinghua University)
CodeRetrievalTransformerImage
π― What it does: A deep factorized metric learning (DFML) framework is proposed, which dynamically allocates training samples to submodules of the Transformer model using learnable routers, achieving diversified feature extraction within the model.
π― What it does: This paper proposes a differentiable top-k selection module for achieving partial graph matching in deep graph matching networks, and presents two attention-based aggregation networks (AFA-I, AFA-U) for adaptive prediction of the number of matching inliers k. Additionally, it reconstructs and releases a visual graph matching benchmark for partial matching, IMC-PT-SparseGM, and provides a sparse implementation of NGMv2.
π― What it does: Designed and implemented an end-to-end deep neural network E2P to directly reconstruct high dynamic range, low motion blur polarization intensity, polarization angle, and polarization degree from PDAVIS event streams.
π― What it does: An improved deep image prior model called Deep Random Projector (DRP) is proposed, which significantly accelerates the optimization process of the original Deep Image Prior (DIP) while maintaining recovery quality by freezing randomly initialized network weights, reducing network depth, and incorporating explicit priors (TV).
π― What it does: This paper presents DeepLSD, a hybrid method that combines the line attraction field learned by deep networks with traditional handcrafted line segment detectors, achieving accurate and robust line segment detection in 'wild' images without real line annotations, and further improving accuracy through optimization.
CodeOptimizationExplainability and InterpretabilityAdversarial AttackImageTextTabular
π― What it does: This paper proposes and proves that in sufficiently trained deep neural networks, reasoning logic can be represented by sparse interactive concepts (interaction patterns) and constructs them as causal graphs and And-Or graphs.
π― What it does: The Dense Distinct Queries (DDQ) framework is proposed, which integrates dense and distinct queries to achieve end-to-end object detection, significantly improving the accuracy of models such as FCN, RβCNN, and DETR.
π― What it does: This paper proposes a general 'Dependency Graph' (DepGraph) method for automatic structural pruning of any network architecture, capable of identifying and simultaneously pruning coupled parameter groups in one go.
Depth Estimation From Indoor Panoramas With Neural Scene Representation
Wenjie Chang (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
CodeDepth EstimationNeural Radiance FieldImage
π― What it does: A framework for indoor panoramic depth estimation based on neural implicit fields is proposed, using two networks (SDF network and color network) to learn scene geometry and color from a small number of panoramic images without the need for depth labels.
DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-Training via Word-Region Alignment
Lewei Yao (Hong Kong University of Science and Technology), Hang Xu (Huawei Noah's Ark Lab)
CodeObject DetectionTransformerVision Language ModelContrastive LearningImageText
π― What it does: This paper presents DetCLIPv2, an end-to-end open vocabulary object detection pre-training framework that directly learns fine-grained word-region alignment from large-scale image-text pairs and achieves unified optimization of detection, localization, and image-text alignment through joint training.
π― What it does: This paper proposes a multi-modal media manipulation detection and localization task (DGM4), constructs a dataset containing 230k samples centered on human news, and introduces the HAMMER model, which incorporates hierarchical contrastive learning and cross-modal attention.
π― What it does: A backdoor trigger detection method based on Test-time Consistency evaluation of image corruption (TeCo) is proposed, which only utilizes black-box hard label outputs without the need for additional data or trigger assumptions.
Detecting Everything in the Open World: Towards Universal Object Detection
Zhenyu Wang (Tsinghua University), Shengjin Wang (Tsinghua University)
CodeObject DetectionConvolutional Neural NetworkVision Language ModelImage
π― What it does: A universal object detection framework named UniDetector is proposed, which can utilize multi-source heterogeneous label spaces for training and directly detect any category in the open world without training samples.
π― What it does: This paper proposes a method for out-of-distribution (OOD) detection of image data by calculating the Hamming distance between test samples and training samples using the binary activation pattern (NAP) in ReLU networks.
π― What it does: A two-stage SDF generation framework based on diffusion models (SDF-Diffusion) is proposed, which first generates low-resolution SDF and then generates high-resolution SDF through super-resolution, and can directly output meshes.
π― What it does: An end-to-end domain adaptive semantic segmentation framework DiGA is proposed through symmetric knowledge distillation, cross-domain mixed data augmentation, and threshold-free bidirectional consensus pseudo-labeling.
π― What it does: A variable-dimensional diffusion process (DVDP) is proposed, which dynamically reduces the signal dimension during the diffusion process to accelerate training and sampling.
π― What it does: A 360Β° image rescaling method called DINN360 is proposed, utilizing reversible networks to achieve deformable downsampling and latitude-aware high-frequency projection, recovering high-resolution 360Β° scenes from low-resolution images.
π― What it does: The research proposes a selection and correction method based on dynamic instance-specific thresholds (DISC), which utilizes dual-view learning to partition and correct data with noisy labels, thereby improving classification performance in noisy environments.
Discovering the Real Association: Multimodal Causal Reasoning in Video Question Answering
Chuanqi Zang (Beijing Institute of Technology), Wei Liang (Beijing Institute of Technology)
CodeRecognitionObject DetectionVision Language ModelVideoTextMultimodality
π― What it does: This paper proposes a multi-modal causal reasoning framework MCR, aimed at eliminating the co-occurrence bias between visual and textual information in video question answering tasks, thereby enhancing the model's robustness and generalization ability.
π― What it does: A framework for facial recognition generalization manifold adversarial attack (GMAA) is proposed, which expands the attack targets to a multi-state set and enhances the attack space from discrete points to a continuous manifold through facial action coding, thereby generating more generalized adversarial samples.
π― What it does: This paper proposes a Structure-Enhanced Recursive Variational Autoencoder (SR-VAE) for unsupervised OOD (Out-of-Distribution) object detection, integrating LoG structural enhancement, recursive VAE to generate diverse classification features, and cyclic consistency conditional VAE to synthesize virtual OOD features.
Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection
Long Li (Northwestern Polytechnical University), Fahad Shahbaz Khan (CVL Linkoping University)
CodeObject DetectionTransformerImage
π― What it does: A Transformer-based DMT framework is proposed to achieve co-salient object detection while explicitly mining co-salient and background information.
π― What it does: A feature map distillation method based on a teacher discriminator (DCD) is proposed, combined with collaborative adversarial training to achieve efficient compression of generators such as CycleGAN and Pix2Pix.
π― What it does: Proposes the Style-Disentangled Transformer (SDT), which achieves online Chinese handwritten character generation through writer-level and character-level style representation, and extends to offline handwritten character generation.
π― What it does: A human pose estimation framework named DistilPose is proposed, which utilizes the knowledge of a heatmap teacher model to enhance the accuracy and efficiency of a regression-based student model.
DistractFlow: Improving Optical Flow Estimation via Realistic Distractions and Pseudo-Labeling
Jisoo Jeong (Qualcomm AI Research), Fatih Porikli (Qualcomm AI Research)
CodeData-Centric LearningOptical FlowImageVideo
π― What it does: A data augmentation method for optical flow training called DistractFlow is proposed, which uses real scene images as distractions to enhance model robustness.
Distribution Shift Inversion for Out-of-Distribution Prediction
Runpeng Yu (National University of Singapore), Xinchao Wang (National University of Singapore)
CodeDomain AdaptationDiffusion modelImage
π― What it does: This paper studies a Distribution Shift Inversion (DSI) algorithm that maps test samples back to the training distribution under unseen distributions to improve Out-of-Distribution (OoD) prediction performance.
π― What it does: A DEEN network is proposed to generate diverse embeddings in the embedding space and aggregate multi-layer features, enhancing visible-infrared portrait recognition performance.
π― What it does: A Diversity-Aware Meta Visual Prompting (DAM-VP) method is proposed, which utilizes clustering to adaptively partition downstream datasets and learn independent prompts for each subset, while using meta-learning pre-trained meta-prompts to accelerate and enhance the transfer performance of frozen pre-trained models on visual tasks.
π― What it does: A Diversified Knowledge Transfer Transformer (DKT) is proposed, which achieves knowledge transfer and catastrophic forgetting suppression in class-incremental learning through task-general and task-specific attention modules and a dual classifier.
π― What it does: A model-agnostic Binary Transformation Layer (BTL) is proposed, which directly learns binary descriptors (DLBD) under a self-supervised framework, and a wide-temperature calibrated cross-entropy loss is designed to enhance the diversity of descriptor distributions.
DNF: Decouple and Feedback Network for Seeing in the Dark
Xin Jin (Nankai University), Chongyi Li (Nanyang Technological University)
CodeRestorationConvolutional Neural NetworkImage
π― What it does: A DNF (Decouple and Feedback Network) framework is proposed for low-light enhancement of RAW images, utilizing domain-specific decoupling and feature-level feedback to achieve more accurate denoising and color recovery.