These 830 CVPR 2023 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every CVPR 2023 paper, free trial on arXivSub.
3D Highlighter: Localizing Regions on 3D Shapes via Text Descriptions
Dale Decatur, Rana Hanocka
CodeObject DetectionSegmentationVision Language ModelMesh
π― What it does: This study proposes 3D Highlighter, a technique for semantic region localization of 3D meshes based on text descriptions;
Shaohui Liu (ETH Zurich), Viktor Larsson (Lund University)
CodeObject DetectionOptimizationSimultaneous Localization and MappingImage
π― What it does: A complete 3D line segment mapping pipeline, LIMAP, is proposed for reconstructing high-quality 3D line segments from multi-view images and generating line-point and line VP association maps.
π― What it does: Generating semantic scene graphs from 3D point clouds, utilizing spatial hierarchical knowledge to assist in relationship prediction.
π― What it does: A two-stage 3D video object detection framework called BA-Det is proposed, which jointly optimizes object detection and pose estimation using learnable long-term visual correspondence and optimizable object-centric bundle adjustment.
π― What it does: This paper proposes Pix2Pix3D, a 3D generative model that maps 2D label maps (such as segmentation maps or edge maps) to renderable 3D neural fields, capable of synthesizing high-quality images from arbitrary viewpoints and generating corresponding pixel-aligned label maps.
π― What it does: This paper proposes a semi-automatic labeling method based on a motion capture (Mo-Cap) system, constructing the first 3D pose and identity labeling dataset for birds (pigeons) called 3D-POP, which includes approximately 300,000 frames and 2 million keypoint instances, covering multiple angles, multiple individuals, and various behaviors.
π― What it does: A data-driven framework is proposed to measure the impact of various classes (or samples) in the source dataset on the performance of downstream tasks in transfer learning, and to improve performance by removing negative classes/samples.
π― What it does: A general regret upper bound for constrained full matrix preconditioning gradients is proposed, and based on this theory, the AdaBK optimizer is designed (which can be embedded into SGDM and AdamW, resulting in SGDM BK and AdamW BK).
A Generalized Framework for Video Instance Segmentation
Miran Heo (Yonsei University), Seon Joo Kim (Yonsei University)
CodeObject DetectionSegmentationTransformerVideo
π― What it does: A general video instance segmentation framework GenVIS is proposed, capable of processing long videos in online and semi-online modes.
A Simple Baseline for Video Restoration With Grouped Spatial-Temporal Shift
Dasong Li (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)
CodeRestorationConvolutional Neural NetworkVideo
π― What it does: A lightweight video restoration framework based on grouped spatial-temporal shifts is designed to achieve video deblurring and denoising without using optical flow, deformable convolution, or self-attention.
π― What it does: A two-stage deep learning method was designed for the instance segmentation of cell bodies (soma) from high-resolution EM data of the complete adult fruit fly brain, and the first cell body annotation dataset (EMADS) was constructed.
π― What it does: A general few-shot semantic segmentation framework called DIaM is proposed. This framework uses standard supervised learning during the training phase and employs optimization methods that maximize mutual information and knowledge distillation for incremental few-shot adaptation of any pre-trained segmentation network during the inference phase.
π― What it does: A lightweight unified pyramid recursive network, UPR-Net, is proposed for video frame interpolation, utilizing forward warping and iterative refinement to achieve high-quality intermediate frame synthesis.
π― What it does: A pluggable prompt tuning method called Aβlaβcarte Prompt Tuning (APT) is proposed, achieving composable sub-data models to meet the needs of continual learning, data forgetting, and user customization;
ABCD: Arbitrary Bitwise Coefficient for De-Quantization
Woo Kyoung Han (Daegu Gyeongbuk Institute of Science and Technology), Kyong Hwan Jin (Daegu Gyeongbuk Institute of Science and Technology)
CodeRestorationImageVideo
π― What it does: An arbitrary bit-depth expansion (ABCD) model based on implicit neural representation is proposed, capable of recovering high bit-depth images from any quantized low bit-depth images, and supports multiple bit depths simultaneously during training.
ABLE-NeRF: Attention-Based Rendering With Learnable Embeddings for Neural Radiance Field
Zhe Jun Tang (Nanyang Technological University), Haiyu Zhao (SenseTime Research)
CodeTransformerNeural Radiance FieldImage
π― What it does: A Transformer-based ABLE-NeRF is designed, utilizing self-attention to simulate volumetric rendering and incorporating learnable embeddings to memorize scene lighting, thereby enhancing perspective-dependent rendering quality.
Abstract Visual Reasoning: An Algebraic Approach for Solving Raven's Progressive Matrices
Jingyi Xu (Singapore University of Technology and Design), Kai Fong Ernest Chong (Singapore University of Technology and Design)
CodeObject DetectionGenerationImage
π― What it does: This paper proposes an 'algebraic machine reasoning' framework based on algebraic ideals, abstracting Ravenβs Progressive Matrices (RPM) problems as ideal operations in polynomial rings, utilizing algebraic subroutines such as GrΓΆbner bases and primary decomposition for answer selection and generation.
Accelerating Vision-Language Pretraining With Free Language Modeling
Teng Wang (Southern University of Science and Technology), Ping Luo (Southern University of Science and Technology)
CodeRetrievalComputational EfficiencyRepresentation LearningTransformerVision Language ModelImageTextMultimodality
π― What it does: Proposed the Free Language Modeling (FLM) objective and the encode-corrupt-predict framework to accelerate visual-language pre-training.
π― What it does: This paper proposes an Attention Collaboration-based Regressor (ACR) that can achieve 3D reconstruction of any two hands from a single RGB image, taking into account complex situations such as hand interactions, truncation, and occlusion.
π― What it does: This paper defines the Active Finetuning task, which aims to select a small subset of samples from a large pool of unlabeled data for supervised finetuning, and proposes the ActiveFT method to achieve this goal.
π― What it does: This paper proposes an online test-time training method called ActMAD based on location-aware activation matching to adapt to distribution shifts in the absence of source data.
π― What it does: An end-to-end trainable adaptive masking strategy called AdaMAE has been developed for video masked autoencoders, achieving efficient spatiotemporal learning.
Adaptive Annealing for Robust Geometric Estimation
Chitturi Sidhartha (Indian Institute of Science), Venu Madhav Govindu (Indian Institute of Science)
CodeAutonomous DrivingOptimizationPoint Cloud
π― What it does: A GNC method based on Hessian positive definiteness adaptive annealing is proposed for robust geometric estimation, particularly for 3D point cloud registration.
π― What it does: This paper proposes AdaMatcher, an end-to-end detector-free feature matching framework that provides adaptive assignment and scale alignment.
π― What it does: Proposes Adaptive Data-Free Quantization (AdaDFQ), which calibrates the quantization network by generating adaptively adjustable synthetic samples without the need for original training data.
π― What it does: An adaptive global decay process is proposed for event stream processing of event cameras, and its effectiveness is validated across various tasks.
Lai Wei (Shanghai Maritime University), Jin Liu (Shanghai Maritime University)
CodeGraph Neural NetworkImage
π― What it does: Adaptive Graph Convolutional Subspace Clustering (AGCSC) is achieved by using graph convolution technology combined with feature extraction and self-expression matrix constraints.
Chung-Ching Lin (Microsoft), Zicheng Liu (Microsoft)
CodeSegmentationTransformerVideo
π― What it does: The AdaM framework is proposed to achieve alpha blending of human foregrounds in dynamic videos without the need for trimaps or pre-collected backgrounds.
Adaptive Patch Deformation for Textureless-Resilient Multi-View Stereo
Yuesong Wang (Huazhong University of Science and Technology), Yawei Luo (Zhejiang University)
CodeDepth EstimationOptimizationImageBenchmark
π― What it does: An adaptive patch deformation is introduced based on traditional PatchMatch, dynamically expanding the receptive field for unreliable pixels in texture-missing areas, and using anchor pixels to ensure that the matching cost approaches global optimality, thereby achieving texture-insensitive and low-memory multi-view stereo reconstruction.
π― What it does: A method for accelerating detection heads based on sparse convolution, called CEASC, is proposed for object detection in drone images.
π― What it does: This paper proposes a Sparse Pairwise (SP) loss and its adaptive version AdaSP for the task of person/vehicle re-identification, where only one pair of positive samples and one pair of negative samples are sampled for each category during training.
π― What it does: A pluggable AdaptiveMix module is proposed, which enhances the training stability of GANs and the quality of generated images by pulling mixed samples closer in the feature space of the discriminator, thereby shrinking the distribution area of training samples.
Adjustment and Alignment for Unbiased Open Set Domain Adaptation
Wuyang Li (City University of Hong Kong), Yixuan Yuan (The Chinese University of Hong Kong)
CodeDomain AdaptationImage
π― What it does: To address the semantic-level bias problem in open set domain adaptation, the ANNA framework is proposed, which discovers potential new category regions in the source domain through Front-Door Adjustment (FDA) to achieve unbiased learning. It then separates and aligns the base classes and new classes through Decoupled Causal Alignment (DCA), ultimately achieving unbiased open set domain adaptation.
Advancing Visual Grounding With Scene Knowledge: Benchmark and Method
Zhihong Chen (Chinese University of Hong Kong), Guanbin Li (Sun Yat-sen University)
CodeRecognitionObject DetectionTransformerVision Language ModelImageTextMultimodalityBenchmark
π― What it does: This work proposes the Scene Knowledge Driven Visual Grounding (SK-VG) task and constructs a corresponding dataset; it also designs two knowledge embedding-based single-stage methods, KeViLI, and a language structure-based two-stage method, LeViLM, to explore the role of scene knowledge in visual-language alignment.
Adversarial Normalization: I Can Visualize Everything (ICE)
Hoyoung Choi (Hanyang University), Kyungsik Han (Hanyang University)
CodeSegmentationExplainability and InterpretabilityAdversarial AttackTransformerImage
π― What it does: Proposes a method to visualize the interpretability of visual Transformers (ICE) by classifying each image patch and using adversarial normalization.
π― What it does: Proposes two methods, Affordance Transformer (Afformer) and Masked Affordance Hand (MaskAHand), to achieve fine-grained interaction (affordance) localization from video to image.
π― What it does: This paper proposes an adversarial training method based on extended attribution span and mixed feature fusion (AGAIN) to enhance the generalization performance of robust models.
Aligning Bag of Regions for Open-Vocabulary Object Detection
Size Wu (Nanyang Technological University), Chen Change Loy (The University of Hong Kong)
CodeObject DetectionConvolutional Neural NetworkVision Language ModelContrastive LearningImageText
π― What it does: The BARON method is proposed, which extends open vocabulary object detection from single region alignment to 'region pack' alignment, utilizing a combined structure of pre-trained visual language models to enhance the performance of new category detection.
All in One: Exploring Unified Video-Language Pre-Training
Jinpeng Wang (National University of Singapore), Mike Zheng Shou (National University of Singapore)
CodeRecognitionRetrievalTransformerVision Language ModelContrastive LearningVideoTextMultimodality
π― What it does: A unified end-to-end video language pre-training model called All-in-one Transformer is proposed, which learns multimodal representations directly from raw video pixels and a text encoder without the need for separate visual/language encoders.
π― What it does: A lightweight MLP-like architecture is proposed, introducing Dynamic Low-Frequency Spectrum Transformation (ALOFT) to enhance domain generalization performance.
Ambiguity-Resistant Semi-Supervised Learning for Dense Object Detection
Chang Liu (Shanghai University), Jingdong Wang (Baidu Inc)
CodeObject DetectionSupervised Fine-TuningImage
π― What it does: This paper addresses the issues of selection ambiguity and assignment ambiguity in single-stage object detectors within semi-supervised learning, proposing a framework named ARSL to alleviate these two types of problems, thereby significantly improving detection performance.
π― What it does: A medical image fuzzy segmentation method based on diffusion models (CIMD) is proposed, which can generate multiple feasible segmentation results from a single image;
π― What it does: This paper proposes the AMT (All-Pairs Multi-Field Transforms) network for efficient video frame interpolation, utilizing bidirectional all-pairs correlation fields and multi-field refinement to achieve more accurate task-oriented optical flow.
π― What it does: Build a single model to jointly train and identify fine-grained labels from multiple fine-grained visual classification datasets without relying on coarse-grained labels.
An In-Depth Exploration of Person Re-Identification and Gait Recognition in Cloth-Changing Conditions
Weijia Li (Beijing Jiaotong University), Yao Zhao (Beijing Jiaotong University)
CodeRecognitionRetrievalVideoBenchmark
π― What it does: A new video ReID and gait recognition benchmark, CCPG, with seven types of clothing variations has been constructed, and methods for video ReID and gait recognition have been compared on it.
π― What it does: This paper proposes Anchor3DLane, a complete end-to-end detection framework that directly regresses 3D lanes through 3D line anchors in front view (FV) features without the need for BEV transformation.
π― What it does: A label transfer framework based on sawtooth annealing scheduling (Annealing-based Label-Transfer) is proposed, enabling the simultaneous learning of known and unknown categories in open-world object detection (OWOD) tasks without manually selecting unknown boxes.
π― What it does: This paper proposes an interactive hierarchical uncertainty active learning framework (HUAL) that uses binary labels instead of complete timestamps for video moment retrieval training, significantly reducing annotation costs.
π― What it does: The ASAP benchmark is proposed to evaluate the performance of camera-based 3D detection in self-driving scenarios under real-time inference (streaming) conditions, and it is extended to 12Hz high frame rate annotations on the nuScenes dataset. Additionally, the SPUR evaluation protocol, speed prediction, and learning-based prediction baseline methods are designed to compensate for inference delays. Comprehensive comparative experiments are conducted on various existing 3D detection models under different GPU and shared resource environments.
ARKitTrack: A New Diverse Dataset for Tracking Using Mobile RGB-D Data
Haojie Zhao (Dalian University of Technology), Huchuan Lu (Dalian University of Technology)
CodeObject TrackingSegmentationTransformerSimultaneous Localization and MappingVideoBenchmark
π― What it does: This paper presents a dataset called ARKitTrack, which consists of 300 RGB-D video sequences (a total of 455 targets and 229.7K frames) collected using the iPhone LiDAR camera. It provides axis-aligned bounding boxes, pixel-level masks, frame-level attributes, and camera intrinsic parameters and poses. Additionally, a unified RGB-D tracking baseline is provided, achieving a closed loop between VOT and VOS.
Attribute-Preserving Face Dataset Anonymization via Latent Code Optimization
Simone Barattin (University of Trento), Nicu Sebe (University of Trento)
CodeRecognitionGenerationSafty and PrivacyGenerative Adversarial NetworkImage
π― What it does: By directly optimizing the latent codes in the pre-trained StyleGAN2 latent space, identity de-identification of facial images is achieved while maintaining the facial attributes of the original images.
Augmentation Matters: A Simple-Yet-Effective Approach to Semi-Supervised Semantic Segmentation
Zhen Zhao (University of Sydney), Jingdong Wang (Baidu Inc.)
CodeSegmentationKnowledge DistillationImage
π― What it does: This paper studies a semi-supervised semantic segmentation method called AugSeg based on a teacher-student framework, focusing on improving data augmentation strategies.
AutoLabel: CLIP-Based Framework for Open-Set Video Domain Adaptation
Giacomo Zara (University of Trento), Elisa Ricci (University of Trento)
CodeClassificationDomain AdaptationTransformerVision Language ModelContrastive LearningVideo
π― What it does: Proposes the AutoLabel framework, which automatically generates candidate unknown category labels for the target domain on the open-source CLIP, thereby achieving open-set domain adaptation for video action classification in the unlabeled target domain.
π― What it does: This paper proposes a fully automated wire segmentation and removal system for high-resolution images, which includes a two-stage coarse-to-fine segmentation network and a LaMa-based tiled image restoration pipeline.
Xing Wei (Xi'an Jiaotong University), Yihong Gong (Xi'an Jiaotong University)
CodeObject TrackingTransformerVideo
π― What it does: ARTrack is proposed, an end-to-end autoregressive visual tracking framework that treats target trajectories as discrete coordinate sequences, directly decoded through a Transformer without the need for traditional localization heads or post-processing.
π― What it does: This paper proposes an arbitrary scale screen content image super-resolution method based on a B-spline texture coefficient estimator (BTC).
π― What it does: This paper proposes a monocular 3D vehicle pose and shape reconstruction framework based on Bi-Contextual Attention and Attention-Guided Modeling (BAAM), and introduces 3D Non-Maximum Suppression (3D NMS) to eliminate false detections;
π― What it does: This paper proposes a motion-blurred image combined bundle-adjustment neural radiance field (BAD-NeRF), which can simultaneously learn 3D scene representation and the motion trajectory during camera exposure, thus achieving deblurring and high-quality view synthesis under severely blurred images.
π― What it does: The paper proposes injecting class-related logit perturbations during the training phase to alleviate the long-tail problem in semantic segmentation.
Batch Model Consolidation: A Multi-Task Model Consolidation Framework
Iordanis Fostiropoulos (University of Southern California), Laurent Itti (University of Southern California)
CodeClassificationKnowledge DistillationConvolutional Neural NetworkMixture of ExpertsImageSequential
π― What it does: A Batch Model Consolidation (BMC) framework is proposed, which achieves long-sequence learning across multiple domains and tasks in continual learning by parallel training multiple expert models and integrating their knowledge into a baseline model through batch distillation at each step.
Behavioral Analysis of Vision-and-Language Navigation Agents
Zijiao Yang (Oregon State University), Stefan Lee (Oregon State University)
CodeTransformerVision Language ModelMultimodality
π― What it does: This paper studies and quantifies the fine-grained behaviors of visual-language navigation agents in four types of skills: stopping, turning, finding objects, and finding rooms, by constructing trajectory truncation and incorporating skill-specific sub-instructions as intervention samples.
π― What it does: A multimodal framework utilizing image and tabular data for self-supervised contrastive learning is proposed, which is fine-tuned on image-only tasks, significantly improving the performance of cardiovascular disease prediction and car model recognition.
π― What it does: This paper proposes a CMOS network for simultaneously estimating spatially varying blur kernels and semantic maps to achieve better blind super-resolution.
π― What it does: A self-supervised learning framework called SOLIDER with adjustable semantic ratios was trained on a large-scale unlabeled human image dataset to learn general human visual representations.
π― What it does: This paper proposes a bidirectional distribution alignment generative model Bi-VAEGAN to enhance the performance of Transductive Zero-Shot Learning (TZSL).
π― What it does: Bi3D is proposedβa dual-domain active learning framework for cross-domain 3D object detection under the condition of having very few labeled target domain samples.
π― What it does: This paper systematically evaluates the impact of neural network pruning on visual model bias and proposes a bias prediction and mitigation method based on dense models.
π― What it does: A BiasBed evaluation platform has been constructed for the systematic comparison and assessment of various algorithms aimed at reducing texture bias in convolutional networks, providing a unified process for experiments, model selection, and statistical testing.
π― What it does: This paper proposes a framework called BiCro based on bidirectional cross-modal similarity consistency to correct noisy correspondences in multimodal matching.
Bidirectional Cross-Modal Knowledge Exploration for Video Recognition With Pre-Trained Vision-Language Models
Wenhao Wu (University of Sydney), Wanli Ouyang (Shanghai AI Laboratory)
CodeRecognitionTransformerVision Language ModelContrastive LearningVideoText
π― What it does: This paper proposes the BIKE framework, which utilizes the bidirectional cross-modal knowledge of the pre-trained vision-language model (CLIP) to enhance video recognition performance, mainly consisting of an attribute branch and a video branch.
π― What it does: In 4K video frame interpolation, BiFormer is proposed, which first roughly estimates the global motion field through a bilateral Transformer, and then refines it to synthesize the intermediate frames.
π― What it does: This paper proposes a Bi-Level Routing Attention (BRA) mechanism and constructs a new visual Transformer backbone network called BiFormer, which is used for tasks such as image classification, object detection, instance segmentation, and semantic segmentation.
BioNet: A Biologically-Inspired Network for Face Recognition
Pengyu Li (Terminus Group)
CodeRecognitionConvolutional Neural NetworkImage
π― What it does: A facial recognition network called BioNet is proposed, which is based on the feature partitioning of the lower temporal cortex of the human brain and integrates attribute knowledge to enhance recognition performance.
π― What it does: This paper proposes a method for recovering JPEG images affected by bit errors in encrypted bitstreams. It first designs a robust decoder to complete the full decoding, and then achieves high-quality recovery within a two-stage compensation and alignment framework (SCA and GCA).
π― What it does: A black-box visual prompt method called BlackVIP is designed and implemented, utilizing an input-adaptive visual prompt generation network (Coordinator) and a zero-order optimization algorithm (SPSA-GC) to achieve transfer learning without accessing pre-trained model parameters and with extremely low memory requirements.
π― What it does: This paper proposes a post-hoc discrete distribution detection framework based on feature map norms, utilizing a block selection method to identify the most suitable network blocks for detection without the need for external OOD samples during training, and using the feature norm of these blocks for OOV discrimination.
π― What it does: An Adaptive Adversarial Distillation (AdaAD) method is proposed, which involves the teacher model in the inner optimization phase to search for points of maximum prediction difference, thereby enhancing both the natural accuracy and adversarial robustness of the student model during the distillation process.
π― What it does: A method for unsupervised pre-training based on visual prompts is proposed to enhance the performance of query-based end-to-end instance segmentation models (such as K-Net) in low-data environments.
π― What it does: This paper proposes a FullMatch framework based on FixMatch, which fully utilizes all unlabeled data to enhance semi-supervised learning performance through Entropy Meaning Loss (EML) and Adaptive Negative Learning (ANL).
π― What it does: Proposes an abstract-based interval training method for robust training of image classification models, accompanied by a black-box verifier.
π― What it does: An unsupervised spatio-temporal correspondence learning framework is designed to enhance the correspondence of matching-based video object segmentation models, thereby improving mask tracking performance.
π― What it does: This paper proposes a weakly supervised temporal action localization framework that utilizes action label text information to achieve two objectives through text-segment mining and video-text language completion, significantly improving localization accuracy.
π― What it does: A new backward-compatible training method (AdvBCT) is proposed, allowing image retrieval systems to upgrade models without recalculating database embeddings.
π― What it does: This paper proposes a box-level active detection framework and designs the Complementary Pseudo Active Strategy (ComPAS) method, which selects the most informative target boxes through an input-end committee and completes sparse annotations using pseudo-labels, thereby reducing annotation redundancy and improving detection performance.
π― What it does: This paper studies weakly supervised instance segmentation and proposes the BoxTeacher framework, which utilizes box annotations to generate high-quality pseudo labels and enhances performance through teacher-student learning.
π― What it does: This paper proposes the InterWild framework, which recovers the 3D mesh of two interacting hands from a single RGB image in the wild.
π― What it does: A point cloud registration framework named BUFFER is proposed, combining point-level and block-level feature learning, while achieving high accuracy, high efficiency, and strong generalization.
π― What it does: This paper studies model stealing attacks on self-supervised image encoders and proposes a stealing method based on contrastive learning called Cont-Steal.
CaPriDe Learning: Confidential and Private Decentralized Learning Based on Encryption-Friendly Distillation Loss
Nurbek Tastan (Mohamed bin Zayed University of Artificial Intelligence), Karthik Nandakumar (Mohamed bin Zayed University of Artificial Intelligence)
CodeFederated LearningSafty and PrivacyKnowledge DistillationConvolutional Neural NetworkImage
π― What it does: A CaPriDe learning framework is proposed, utilizing fully homomorphic encryption (FHE) to achieve decentralized collaborative learning with data confidentiality, and transferring model knowledge through knowledge distillation.
Category Query Learning for Human-Object Interaction Classification
Chi Xie (Tongji University), Yichen Wei (Shanghai Jiao Tong University)
CodeClassificationTransformerImage
π― What it does: Proposes a HOI classification method based on category query learning, mapping category queries to images and enhancing interaction classification performance by combining a transformer decoder.
π― What it does: This study investigates the sensitivity of different categories to adversarial training configurations and proposes an adaptive category-level adversarial training framework called CFA.
Jianlong Wu (Harbin Institute of Technology), Liqiang Nie (Shandong University)
CodeClassificationContrastive LearningImage
π― What it does: This paper proposes a semi-supervised learning framework named CHMatch, which combines instance-level prediction matching with hierarchical label-based graph-level contrastive learning, aiming to improve classification performance under few labeled samples.