CVPR 2023 Papers — Page 12
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2353 papers
Learning Attribute and Class-Specific Representation Duet for Fine-Grained Fashion Analysis
Yang Jiao (Amazon), Yi Sun (Amazon)
RetrievalConvolutional Neural NetworkImage
🎯 What it does: M3-Net is proposed, a network for fine-grained fashion retrieval that jointly learns attribute-level and category-level fashion representations.
Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning
Weixuan Sun (Australian National University), Nick Barnes (Australian National University)
RecognitionObject DetectionSegmentationRepresentation LearningConvolutional Neural NetworkContrastive LearningMultimodalityAudio
🎯 What it does: This paper proposes an audio-visual source localization method based on False Negative Aware Contrastive Learning (FNAC), which utilizes a unimodal similarity matrix to identify potential false negative samples and suppress their impact through two regularization techniques: False Negatives Suppression (FNS) and True Negatives Enhancement (TNE), thereby enhancing the audio-visual correspondence representation.
Learning Bottleneck Concepts in Image Classification
Bowen Wang (Osaka University), Hajime Nagahara (Osaka University)
ClassificationContrastive LearningImage
🎯 What it does: This paper proposes BotCL (Bottleneck Concept Learner), a model that learns interpretable concepts and performs image classification using a slot attention mechanism and self-supervised adversarial learning under the supervision of no concept labels.
Learning Common Rationale To Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems
Yangyang Shu (University of Adelaide), Lingqiao Liu (University of Adelaide)
ClassificationRecognitionRetrievalRepresentation LearningContrastive LearningImage
🎯 What it does: This paper proposes an additional filtering mechanism (Common Rationale Detector) in self-supervised learning, which enhances fine-grained feature representation by fitting GradCAM and filtering out patterns that are useless for fine-grained visual recognition through limited fitting capacity.
Learning Compact Representations for LiDAR Completion and Generation
Yuwen Xiong (Waabi), Raquel Urtasun (Massachusetts Institute of Technology)
Object DetectionGenerationData SynthesisAutonomous DrivingTransformerAuto EncoderGenerative Adversarial NetworkPoint Cloud
🎯 What it does: This paper proposes the UltraLiDAR framework, which utilizes a discretizable LiDAR representation to achieve the densification of sparse point clouds, point cloud generation, and controllable operations on point clouds, significantly enhancing downstream detection performance and generating highly realistic LiDAR scenes.
Learning Conditional Attributes for Compositional Zero-Shot Learning
Qingsheng Wang (Northwestern Polytechnical University), Chunhua Shen (Zhejiang University)
ClassificationRecognitionImage
🎯 What it does: This study investigates conditional attribute learning to enhance the recognition performance of Combinatorial Zero-Shot Learning (CZSL).
Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares
Dominik Muhle (Technical University of Munich), Daniel Cremers (Technical University of Munich)
Pose EstimationAutonomous DrivingOptimizationSimultaneous Localization and MappingImage
🎯 What it does: A differentiable nonlinear least squares (DNLS) framework is proposed to achieve more accurate relative pose estimation by regressing the uncertainty of the position information of corresponding image features.
Learning Customized Visual Models With Retrieval-Augmented Knowledge
Haotian Liu (University of Wisconsin Madison), Chunyuan Li (Microsoft)
Object DetectionSegmentationRetrievalTransformerContrastive LearningImageMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes the REACT framework, which utilizes retrieval-enhanced external visual-textual knowledge to customize visual models, avoiding large-scale pre-training and improving performance in the target domain;
Learning Debiased Representations via Conditional Attribute Interpolation
Yi-Kai Zhang (Nanjing University), Han-Jia Ye (Nanjing University)
ClassificationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: A two-stage χ² model is proposed, which learns debiased representations by mining intermediate attribute samples (IASs) from the training dynamics and performing conditional attribute interpolation to remove spurious associations in biased datasets.
Learning Decorrelated Representations Efficiently Using Fast Fourier Transform
Yutaro Shigeto (Chiba Institute of Technology), Akikazu Takeuchi (Chiba Institute of Technology)
Object DetectionComputational EfficiencyRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes a decoupled regularization method based on Fast Fourier Transform (FFT) that efficiently decorrelates features in self-supervised visual representation learning, reducing the time complexity to O(n d log d) and significantly lowering computational costs.
Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis From Monocular Image
Yu Deng, Heung-Yeung Shum
GenerationData SynthesisPose EstimationGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a unidirectional forward network based on 3D-aware GAN GRAM called GRAMInverter, which can reconstruct high-fidelity, 3D-compatible voxel representations from a single portrait image, and achieve detail preservation and viewpoint synthesis under continuous perspectives through detail-level refinement.
Learning Discriminative Representations for Skeleton Based Action Recognition
Huanyu Zhou (Beihang University), Yunhong Wang (Beihang University)
RecognitionRepresentation LearningGraph Neural NetworkContrastive LearningVideo
🎯 What it does: Proposes a feature refinement head (FR Head) based on contrastive learning to enhance feature representation in skeleton action recognition models, particularly improving the distinction of ambiguous actions.
Learning Distortion Invariant Representation for Image Restoration From a Causality Perspective
Xin Li (University of Science and Technology of China), Zhibo Chen (University of Science and Technology of China)
RestorationRepresentation LearningMeta LearningConvolutional Neural NetworkTransformerImage
🎯 What it does: A distortion-invariant representation learning (DIL) training strategy based on a causal perspective is proposed to enhance the generalization performance of image restoration models on unknown distortions.
Learning Dynamic Style Kernels for Artistic Style Transfer
Wenju Xu, Yongwei Nie
Image TranslationConvolutional Neural NetworkTransformerGenerative Adversarial NetworkImageVideo
🎯 What it does: A framework for arbitrary style transfer based on dynamic style kernels (Style Kernel) is proposed, utilizing content-style alignment attention, a content gating module, and separable convolutions to achieve pixel-level adaptive style transfer.
Learning Emotion Representations From Verbal and Nonverbal Communication
Sitao Zhang (Pennsylvania State University), James Z. Wang (Pennsylvania State University)
Representation LearningTransformerContrastive LearningVideoTextMultimodality
🎯 What it does: EmotionCLIP is constructed, a framework for visual emotion representation pre-training through unlabeled everyday communication videos and subtitles.
Learning Event Guided High Dynamic Range Video Reconstruction
Yixin Yang (Peking University), Boxin Shi (Peking University)
RestorationGenerationConvolutional Neural NetworkRecurrent Neural NetworkVideoMultimodality
🎯 What it does: This study proposes a multi-modal learning-based HDRev-Net framework that jointly reconstructs high dynamic range (HDR) videos using LDR videos and event streams from event cameras.
Learning Expressive Prompting With Residuals for Vision Transformers
Rajshekhar Das (Carnegie Mellon University), Ashwin Swaminathan (AWS AI Labs)
ClassificationSegmentationTransformerPrompt EngineeringImage
🎯 What it does: A new visual Transformer adaptation method called EXPRES is proposed, which achieves efficient downstream task transfer in a frozen ViT through shallow prompts and residual prompts.
Learning Federated Visual Prompt in Null Space for MRI Reconstruction
Chun-Mei Feng (Agency for Science Technology and Research), Wangmeng Zuo (Harbin Institute of Technology)
RestorationFederated LearningTransformerPrompt EngineeringBiomedical DataMagnetic Resonance Imaging
🎯 What it does: The FedPR algorithm is proposed, which learns distributed visual prompts in the approximate null space of global prompts to address the issues of communication costs, data scarcity, and catastrophic forgetting in federated MRI reconstruction.
Learning From Noisy Labels With Decoupled Meta Label Purifier
Yuanpeng Tu (Tongji University), Cai Rong Zhao (Tongji University)
Representation LearningMeta LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes a multi-stage label purification method called Decoupled Meta Label Purifier (DMLP) to enhance the robustness of deep networks in the presence of noisy labels.
Learning From Unique Perspectives: User-Aware Saliency Modeling
Shi Chen (University of Minnesota), Junfeng He (Google)
Convolutional Neural NetworkImage
🎯 What it does: A user-aware attention modeling framework is proposed, capable of predicting the visual attention distribution of individuals, user groups, and the overall population simultaneously.
Learning Generative Structure Prior for Blind Text Image Super-Resolution
Xiaoming Li (Nanyang Technological University), Chen Change Loy (Nanyang Technological University)
RestorationSuper ResolutionTransformerGenerative Adversarial NetworkImage
🎯 What it does: This study focuses on blind text image super-resolution and proposes the use of generative structural priors to guide the restoration.
Learning Geometric-Aware Properties in 2D Representation Using Lightweight CAD Models, or Zero Real 3D Pairs
Pattaramanee Arsomngern (Vistec), Supasorn Suwajanakorn (Vistec)
Object DetectionSegmentationRetrievalGraph Neural NetworkContrastive LearningImagePoint Cloud
🎯 What it does: This paper proposes a method to construct a unified 2D-3D embedding space using lightweight CAD models, and enhances the performance of 2D scene understanding tasks through geometric-aware attribute pre-training.
Learning Geometry-Aware Representations by Sketching
Hyundo Lee (Seoul National University), Byoung-Tak Zhang (Seoul National University)
RetrievalDomain AdaptationRepresentation LearningTransformerContrastive LearningImage
🎯 What it does: A method for learning geometric perception visual representations through generating abstract sketches is proposed.
Learning Human Mesh Recovery in 3D Scenes
Zehong Shen (Zhejiang University), Xiaowei Zhou (Zhejiang University)
Pose EstimationConvolutional Neural NetworkTransformerPoint CloudMesh
🎯 What it does: The study achieves the recovery of absolute human pose and shape from a single image in a pre-scanning scenario, enabling a single forward inference.
Learning Human-to-Robot Handovers From Point Clouds
Sammy Christen (ETH Zurich), Yu-Wei Chao (NVIDIA)
Robotic IntelligenceReinforcement LearningPoint Cloud
🎯 What it does: A point cloud-based visual learning framework is proposed, achieving end-to-end learning from perception to action by training the control strategy of human-robot handover in the HandoverSim environment.
Learning Imbalanced Data With Vision Transformers
Zhengzhuo Xu (Shenzhen International Graduate School Tsinghua University), Chun Yuan (Shenzhen International Graduate School Tsinghua University)
ClassificationRecognitionTransformerSupervised Fine-TuningImage
🎯 What it does: This study investigates how to train Vision Transformers (ViT) from scratch for long-tail recognition on extremely imbalanced data.
Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-Commerce
Yang Jin (Peking University), Yadong Mu (Peking University)
ClassificationObject DetectionRetrievalRecommendation SystemTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A multi-modal pre-training model for e-commerce scenarios, ECLIP, is proposed, focusing on learning instance-level visual-semantic representations and pre-training on a massive dataset of product-image pairs in an unsupervised manner.
Learning Joint Latent Space EBM Prior Model for Multi-Layer Generator
Jiali Cui (Stevens Institute of Technology), Tian Han (University of California)
GenerationAnomaly DetectionAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: A joint energy-based model (EBM) is proposed as a prior for multi-layer generative models, capturing intra-layer contextual relationships and inter-layer structures by jointly modeling the energies of all latent variables.
Learning Locally Editable Virtual Humans
Hsuan-I Ho (ETH Zurich), Otmar Hilliges (ETH Zurich)
GenerationData SynthesisPose EstimationAuto EncoderGenerative Adversarial NetworkMesh
🎯 What it does: This paper proposes a hybrid representation and autoencoder network for generating and editing high-quality 3D virtual humans that are globally consistent in posture.
Learning Multi-Modal Class-Specific Tokens for Weakly Supervised Dense Object Localization
Lian Xu (University of Western Australia), Dan Xu (Hong Kong University of Science and Technology)
Object DetectionTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: This paper proposes a multimodal category-specific tagging framework based on Transformer, which jointly learns CLIP text and visual features to accomplish weakly supervised dense object localization tasks.
Learning Neural Duplex Radiance Fields for Real-Time View Synthesis
Ziyu Wan (City University of Hong Kong), Jing Liao (City University of Hong Kong)
Data SynthesisComputational EfficiencyKnowledge DistillationNeural Radiance FieldImage
🎯 What it does: Learn and distill existing NeRF models to construct a neural dual radiance field based on a dual-layer grid, achieving high-quality real-time novel view synthesis.
Learning Neural Parametric Head Models
Simon Giebenhain (Technical University of Munich), Matthias Nießner (Technical University of Munich)
GenerationData SynthesisNeural Radiance FieldAuto EncoderPoint CloudMesh
🎯 What it does: A neural parametric head model based on neural fields is proposed, capable of representing complete head geometry in decoupled identity and expression latent spaces.
Learning Neural Proto-Face Field for Disentangled 3D Face Modeling in the Wild
Zhenyu Zhang (Tencent Youtu Lab), Chengjie Wang (Tencent Youtu Lab)
GenerationData SynthesisNeural Radiance FieldGenerative Adversarial NetworkImage
🎯 What it does: Proposed the Neural Proto-face Field (NPF) method, achieving unsupervised robust 3D face modeling using multi-image priors.
Learning Neural Volumetric Representations of Dynamic Humans in Minutes
Chen Geng (Zhejiang University), Xiaowei Zhou (Zhejiang University)
GenerationPose EstimationComputational EfficiencyRepresentation LearningNeural Radiance FieldVideo
🎯 What it does: In sparse multi-view videos, dynamic human neural volumetric representation can be learned quickly, generating photorealistic free-viewpoint rendering in about 5 minutes of training.
Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection
Chuangchuang Tan (Beijing Jiaotong University), Yunchao Wei (Beijing Jiaotong University)
ClassificationGenerationAnomaly DetectionConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: The study proposes a method that utilizes a pre-trained CNN model to convert images into gradient representations, and then uses a classifier to detect GAN-generated images.
Learning Open-Vocabulary Semantic Segmentation Models From Natural Language Supervision
Jilan Xu (Fudan University), Weidi Xie (Shanghai Jiao Tong University)
SegmentationTransformerContrastive LearningImageText
🎯 What it does: A Transformer-based open vocabulary semantic segmentation model, OVSegmentor, is proposed and trained, which only utilizes image-text pairs captured by the network without any pixel-level mask labels, enabling zero-shot transfer to various segmentation benchmarks.
Learning Optical Expansion From Scale Matching
Han Ling (Nanjing University of Science and Technology), Zhenwen Ren (Southwest University of Science and Technology)
Depth EstimationAutonomous DrivingOptical FlowImageVideoBenchmark
🎯 What it does: This paper proposes a 3D optical flow estimation method that integrates optical magnification with optical flow, and achieves multi-scale matching through the TPCV module, significantly improving the performance of tasks such as optical flow, depth motion, time-to-collision, and scene flow.
Learning Orthogonal Prototypes for Generalized Few-Shot Semantic Segmentation
Sun-Ao Liu (University of Science and Technology of China), Ting Yao (University of Science and Technology of China)
SegmentationImage
🎯 What it does: This paper proposes the Projection onto Orthogonal Prototypes (POP) framework for Generalized Few-Shot Semantic Segmentation (GFSS), which achieves the separation of base class and novel class features by learning a set of orthogonal prototypes, only updating the novel class prototypes and representing the background with projection residuals.
Learning Partial Correlation Based Deep Visual Representation for Image Classification
Saimunur Rahman (Data61 CSIRO), Changming Sun (Data61 CSIRO)
ClassificationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a sparse inverse covariance estimation layer (iSICE) that can be trained end-to-end for CNN learning of partially correlated visual representations.
Learning Personalized High Quality Volumetric Head Avatars From Monocular RGB Videos
Ziqian Bai (Google), Yinda Zhang (Simon Fraser University)
GenerationData SynthesisPose EstimationConvolutional Neural NetworkNeural Radiance FieldVideo
🎯 What it does: Using single-shot RGB video to train a NeRF model anchored to 3DMM, generating high-quality 3D head avatars with controllable expressions and poses.
Learning Procedure-Aware Video Representation From Instructional Videos and Their Narrations
Yiwu Zhong (University of Wisconsin Madison), Yin Li (Meta AI)
ClassificationRepresentation LearningTransformerDiffusion modelVideoText
🎯 What it does: Learn video representations that can simultaneously encode action steps and their temporal order from a large number of tutorial videos and their voiceovers through unsupervised learning;
Learning Rotation-Equivariant Features for Visual Correspondence
Jongmin Lee (Pohang University of Science and Technology), Minsu Cho (Pohang University of Science and Technology)
RecognitionPose EstimationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: Using self-supervised group-equivariant convolutional networks (group-equivariant CNN) to extract local features, and obtaining rotation-invariant and discriminative descriptors through a 'group-aligning' operation.
Learning Sample Relationship for Exposure Correction
Jie Huang (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
RestorationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a batch sample relationship learning framework (ERL) that enhances the ability of a single network to simultaneously correct two types of exposure errors by learning the exposure-related relationships between underexposed and overexposed images in a batch.
Learning Semantic Relationship Among Instances for Image-Text Matching
Zheren Fu (University of Science and Technology of China), Yongdong Zhang (University of Science and Technology of China)
RetrievalGraph Neural NetworkImageText
🎯 What it does: A hierarchical relationship modeling framework (HREM) is proposed to improve image-text matching.
Learning Semantic-Aware Disentangled Representation for Flexible 3D Human Body Editing
Xiaokun Sun (Tianjin University), Kun Li (Tianjin University)
GenerationPose EstimationRepresentation LearningAuto EncoderMesh
🎯 What it does: Under unsupervised conditions, a fine-grained semantic decoupling method for 3D human representation is proposed, which can be used for precise editing of human body shape and posture.
Learning Semantic-Aware Knowledge Guidance for Low-Light Image Enhancement
Yuhui Wu (University of Electronic Science and Technology of China), Heng Tao Shen (Nanyang Technological University)
RestorationConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A semantic-aware knowledge-guided framework (SKF) is proposed to enhance the quality of low-light image enhancement.
Learning Situation Hyper-Graphs for Video Question Answering
Aisha Urooj, Mubarak Shah (University of Central Florida)
TransformerVideo
🎯 What it does: Using the Transformer architecture to learn contextual hypergraphs (including actions and entity relationships) in videos, and reasoning for video question answering tasks through this hypergraph.
Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution
Yunfan Lu (Hong Kong University of Science and Technology), Lin Wang (Hong Kong University of Science and Technology)
RestorationSuper ResolutionTransformerVideo
🎯 What it does: This paper proposes a joint implicit neural representation framework based on event cameras and RGB frames for achieving video super-resolution at arbitrary scales.
Learning Steerable Function for Efficient Image Resampling
Jiacheng Li (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
RestorationSuper ResolutionConvolutional Neural NetworkImage
🎯 What it does: A new image resampling method called LeRF is proposed, which utilizes deep neural networks to predict variable steerable Gaussian resampling functions and achieves efficient inference through a lookup table (LUT), allowing for continuous resampling under arbitrary geometric transformations.
Learning the Distribution of Errors in Stereo Matching for Joint Disparity and Uncertainty Estimation
Liyan Chen (Stevens Institute of Technology), Philippos Mordohai (Stevens Institute of Technology)
Depth EstimationDomain AdaptationAutonomous DrivingConvolutional Neural NetworkImage
🎯 What it does: A joint estimation of disparity and uncertainty deep stereo matching network called SEDNet is proposed, which enhances the accuracy of disparity and the reliability of uncertainty estimation by constraining the distributions of both using KL divergence.
Learning To Detect and Segment for Open Vocabulary Object Detection
Tao Wang (Sichuan University)
Object DetectionSegmentationConvolutional Neural NetworkVision Language ModelImage
🎯 What it does: Proposes the CondHead dynamic conditional network head, which utilizes semantic embeddings for category-specific parameterization of bounding box regression and mask segmentation, achieving open vocabulary object detection and segmentation.
Learning To Detect Mirrors From Videos via Dual Correspondences
Jiaying Lin (City University of Hong Kong), Rynson W.H. Lau (East China Normal University)
Object DetectionSegmentationConvolutional Neural NetworkVideo
🎯 What it does: The video mirroring detection (VMD) problem is proposed, and the first large-scale video mirroring dataset VMD-D is provided, along with the construction of the VMD-Net model that can simultaneously utilize both intra-frame and inter-frame correspondences.
Learning To Dub Movies via Hierarchical Prosody Models
Gaoxiang Cong (Shandong University), Qingming Huang (Institute of Computing Technology, Chinese Academy of Sciences)
GenerationData SynthesisTransformerVideoTextMultimodalityAudio
🎯 What it does: This paper proposes a hierarchical prosody modeling network for movie dubbing tasks, which achieves high-quality, emotionally synchronized dubbing synthesis by mapping three types of visual information—lip movements, facial emotional expressions (emotional values and arousal in the emotional dimension), and scene atmosphere—onto the duration, energy, pitch, and emotion of speech.
Learning To Exploit Temporal Structure for Biomedical Vision-Language Processing
Shruthi Bannur (Microsoft Health Futures), Ozan Oktay (Microsoft Health Futures)
GenerationRepresentation LearningConvolutional Neural NetworkTransformerVision Language ModelContrastive LearningImageTextMultimodalityBiomedical Data
🎯 What it does: Proposes BioViL-T, which aligns frontal and lateral chest X-ray images with reports through a multi-image encoder, thereby enhancing visual-language representation by utilizing clinical time series information during the pre-training phase.
Learning To Exploit the Sequence-Specific Prior Knowledge for Image Processing Pipelines Optimization
Haina Qin (Institute of Automation, Chinese Academy of Sciences), Weiming Hu (Institute of Automation, Chinese Academy of Sciences)
Object DetectionSegmentationOptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a sequential ISP hyperparameter prediction framework that recursively predicts hyperparameters between ISP modules using convolutional neural networks, and clusters parameters through global similarity grouping to enhance prediction efficiency.
Learning To Fuse Monocular and Multi-View Cues for Multi-Frame Depth Estimation in Dynamic Scenes
Rui Li (Northwestern Polytechnical University), Yanning Zhang (Northwestern Polytechnical University)
Depth EstimationDomain AdaptationAutonomous DrivingConvolutional Neural NetworkSimultaneous Localization and MappingImage
🎯 What it does: A depth estimation framework that integrates monocular and multi-frame visual information is proposed, utilizing a cross-thread fusion module to achieve more accurate depth predictions in dynamic scenes.
Learning To Generate Image Embeddings With User-Level Differential Privacy
Zheng Xu (Google Research), H. Brendan McMahan (Google Research)
Federated LearningSafty and PrivacyRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an algorithm called DP-FedEmb, which utilizes user-level differential privacy (DP) to train image embedding models, focusing on addressing the balance between privacy and efficiency under large class spaces and large model parameter scales.
Learning To Generate Language-Supervised and Open-Vocabulary Scene Graph Using Pre-Trained Visual-Semantic Space
Yong Zhang (Chinese University of Hong Kong), Chang-Wen Chen (Hong Kong Polytechnic University)
Object DetectionGenerationTransformerVision Language ModelImageText
🎯 What it does: This paper proposes the use of a pre-trained Visual-Semantic Space (VSS) to achieve language supervision and open vocabulary scene graph generation, designing the VS3 model and obtaining weakly supervised scene graphs through semantic parsing.
Learning To Generate Text-Grounded Mask for Open-World Semantic Segmentation From Only Image-Text Pairs
Junbum Cha (Kakao Brain), Byungseok Roh (Kakao Brain)
SegmentationConvolutional Neural NetworkTransformerContrastive LearningImageText
🎯 What it does: A Text-Grounded Contrastive Learning (TCL) framework is proposed, which utilizes data containing only image-text pairs and directly incorporates a text localization step into contrastive learning to learn region-text alignment, thereby achieving open-world semantic segmentation.
Learning To Measure the Point Cloud Reconstruction Loss in a Representation Space
Tianxin Huang (Zhejiang University), Yong Liu (Zhejiang University)
ClassificationRestorationRepresentation LearningGenerative Adversarial NetworkContrastive LearningPoint Cloud
🎯 What it does: A Contrastive Adversarial Loss (CALoss) is proposed, which dynamically measures point cloud reconstruction errors in high-dimensional representation space and guides network training.
Learning To Name Classes for Vision and Language Models
Sarah Parisot (Huawei), Steven McDonagh (Huawei)
ClassificationObject DetectionTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: To address the issue of extreme sensitivity to class names in visual-language models, this paper proposes a method of 'learning optimal class names' by learning word vectors for each category based on visual content, replacing manually designed class names, maintaining the capability of an open vocabulary, and achieving seamless adaptation to new datasets.
Learning To Predict Scene-Level Implicit 3D From Posed RGBD Data
Nilesh Kulkarni (University of Michigan), David F. Fouhey (University of Michigan)
SegmentationDepth EstimationNeural Radiance FieldImagePoint Cloud
🎯 What it does: This study proposes an implicit function model trained solely on RGBD images with known poses, capable of reconstructing a complete 3D scene from a single RGB image.
Learning To Render Novel Views From Wide-Baseline Stereo Pairs
Yilun Du (Massachusetts Institute of Technology), Vincent Sitzmann (Massachusetts Institute of Technology)
GenerationData SynthesisConvolutional Neural NetworkTransformerImage
🎯 What it does: A method for novel view rendering using a single wide baseline stereo image pair is proposed.
Learning To Retain While Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation
Gaurav Patel (Purdue University), Qiang Qiu (Purdue University)
Knowledge DistillationMeta LearningGenerative Adversarial NetworkImage
🎯 What it does: A student network update strategy based on meta-learning is proposed, which simultaneously retains learned knowledge and acquires new knowledge in adversarial zero-data knowledge distillation.
Learning To Segment Every Referring Object Point by Point
Mengxue Qu (Beijing Jiaotong University), Yao Zhao (Beijing Jiaotong University)
Object DetectionSegmentationTransformerImage
🎯 What it does: A new Partial-RES (Partial Supervised Reference Expression Segmentation) framework is proposed, which utilizes a large number of bounding box annotations and a very small number of pixel-level masks to train a model that can accurately segment the objects referred to by language expressions.
Learning To Zoom and Unzoom
Chittesh Thavamani (Carnegie Mellon University), Deva Ramanan (Carnegie Mellon University)
Object DetectionSegmentationAutonomous DrivingImage
🎯 What it does: A framework LZU is proposed that first scales and then unscales the image input to achieve efficient sampling while maintaining spatial task performance.
Learning Transferable Spatiotemporal Representations From Natural Script Knowledge
Ziyun Zeng (Tsinghua University), Yixiao Ge (Tencent)
RetrievalRepresentation LearningTransformerContrastive LearningVideoText
🎯 What it does: Using the natural speech transcription text (ASR script) in videos as semantic supervision, a novel self-supervised pre-training task is proposed—Turning to Video for Transcript Sorting (TVTS), which learns transferable spatiotemporal representations of videos by sorting scrambled transcription texts.
Learning Transformation-Predictive Representations for Detection and Description of Local Features
Zihao Wang (Peking University), Zhen Li (Beijing Institute of Technology)
Object DetectionRepresentation LearningTransformerContrastive LearningImage
🎯 What it does: This study investigates a local feature detection and description method that uses only positive samples and self-supervised contrastive learning.
Learning Transformations To Reduce the Geometric Shift in Object Detection
Vidit Vidit (EPFL), Mathieu Salzmann (EPFL)
Object DetectionDomain AdaptationAutonomous DrivingConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an unsupervised domain adaptation method that reduces geometric shifts in object detection by learning a set of homographies, thereby improving cross-domain performance.
Learning Video Representations From Large Language Models
Yue Zhao (Meta AI), Rohit Girdhar (Meta AI)
Representation LearningTransformerLarge Language ModelVision Language ModelContrastive LearningVideoText
🎯 What it does: Utilizing large language models to automatically generate video descriptions, providing dense and diverse textual annotations for long videos, and performing contrastive learning based on these self-generated texts and video features, resulting in a new video-language representation model called LAVILA.
Learning Visibility Field for Detailed 3D Human Reconstruction and Relighting
Ruichen Zheng (Tsinghua University), Tao Yu (Tsinghua University)
RestorationGenerationPose EstimationImagePoint Cloud
🎯 What it does: Proposes a sparse view RGB-D human 3D reconstruction and visibility field learning method, supporting self-shadow relighting.
Learning Visual Representations via Language-Guided Sampling
Mohamed El Banani (University of Michigan), Justin Johnson (University of Michigan)
Representation LearningTransformerLarge Language ModelContrastive LearningImageText
🎯 What it does: Utilizing pre-trained language models to calculate the semantic similarity of image captions, selecting semantically similar image pairs to replace traditional image augmentation or visual nearest neighbors for contrastive learning training of visual representations.
Learning Weather-General and Weather-Specific Features for Image Restoration Under Multiple Adverse Weather Conditions
Yurui Zhu (University of Science and Technology of China), Xiaowei Hu (Shanghai Artificial Intelligence Laboratory)
RestorationConvolutional Neural NetworkImage
🎯 What it does: A unified multi-weather image restoration model has been designed and implemented, which learns weather-general and specific features through two-stage training, and for the first time constructs a real multi-weather dataset.
Learning With Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning
Zeyin Song (Peking University), Yonghong Tian (Peking University)
ClassificationRecognitionContrastive LearningImage
🎯 What it does: Proposes the Semantic-Aware Virtual Contrastive (SAVC) framework, which introduces virtual classes for supervised contrastive learning during the base training phase of Few-Shot Class-Incremental Learning (FSCIL) to enhance inter-class separation while retaining sufficient space for subsequent new classes.
Learning With Noisy Labels via Self-Supervised Adversarial Noisy Masking
Yuanpeng Tu (Tongji University), Cai Rong Zhao (Tongji University)
ClassificationData-Centric LearningGenerative Adversarial NetworkImage
🎯 What it does: A self-supervised adversarial noise masking method (SANM) is proposed, which regularizes the network by adaptively generating masking regions based on label quality to prevent overfitting on data with noisy labels, and utilizes a self-supervised reconstruction task to provide noise-independent supervisory signals.
LEGO-Net: Learning Regular Rearrangements of Objects in Rooms
Qiuhong Anna Wei (Brown University), Leonidas Guibas (Stanford University)
GenerationData SynthesisTransformerScore-based ModelAuto EncoderPoint Cloud
🎯 What it does: A model named LEGO-Net is proposed for the regularized rearrangement of objects in existing indoor scenes, maintaining the original layout style while minimizing object movement distance.
LEMaRT: Label-Efficient Masked Region Transform for Image Harmonization
Sheng Liu (Amazon Prime Video), Raffay Hamid (Amazon Prime Video)
Image HarmonizationRestorationTransformerImage
🎯 What it does: A label-free self-supervised pre-training method called LEMaRT is proposed for the pre-training of image harmonization models.
Less Is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation
Li Li (Durham University), Toby P. Breckon (Durham University)
SegmentationAutonomous DrivingConvolutional Neural NetworkContrastive LearningPoint Cloud
🎯 What it does: This paper proposes a low-parameter, low-label semi-supervised 3D LiDAR point cloud semantic segmentation framework called LiM3D, which can maintain or even improve segmentation accuracy in scenarios with scarce annotations.
Level-S$^2$fM: Structure From Motion on Neural Level Set of Implicit Surfaces
Yuxi Xiao (Wuhan University), Gui-Song Xia (Wuhan University)
Pose EstimationDepth EstimationNeural Radiance FieldSimultaneous Localization and MappingOptical FlowImagePoint Cloud
🎯 What it does: A system for incremental structured light flow based on neural implicit surfaces (Level-S2fM) is proposed, capable of simultaneously estimating camera pose and scene geometry from uncalibrated image collections.
Leverage Interactive Affinity for Affordance Learning
Hongchen Luo (University of Science and Technology of China), Dacheng Tao (JD Explore Academy Institute of Artificial Intelligence)
Object DetectionSegmentationPose EstimationGraph Neural NetworkTransformerImage
🎯 What it does: A posture-guided interactive affordance learning framework is proposed, which utilizes an interaction feature enhancement module and keypoint-driven perception to achieve precise prediction of object usability regions, and constructs a CAL dataset with 5,258 images.
Leveraging Hidden Positives for Unsupervised Semantic Segmentation
Hyun Seok Seong (Sungkyunkwan University), Jae-Pil Heo (Sungkyunkwan University)
SegmentationTransformerContrastive LearningImage
🎯 What it does: This paper proposes a contrastive learning method for unsupervised semantic segmentation using hidden positive samples.
Leveraging Inter-Rater Agreement for Classification in the Presence of Noisy Labels
Maria Sofia Bucarelli (Sapienza University of Rome), Fabrizio Silvestri (Sapienza University of Rome)
ClassificationImage
🎯 What it does: This study investigates how to utilize the consistency statistics among multiple annotators to estimate the label noise distribution and train robust models on classification data with noisy labels.
Leveraging per Image-Token Consistency for Vision-Language Pre-Training
Yunhao Gou (Southern University of Science and Technology), Mingxuan Wang (ByteDance AI Lab)
RetrievalTransformerVision Language ModelImageText
🎯 What it does: This paper proposes a method that utilizes an image-word consistency task (EPIC) to enhance the training objectives of visual-language pre-training.
Leveraging Temporal Context in Low Representational Power Regimes
Camilo L. Fosco (Massachusetts Institute of Technology), Aude Oliva (Massachusetts Institute of Technology)
ClassificationRecognitionConvolutional Neural NetworkTransformerVideo
🎯 What it does: Utilizing the Event Transition Matrix (ETM) obtained from the statistical analysis of action labels in the training set as an auxiliary supervisory signal, a low-parameter video action recognition/prediction model is trained to learn the typical sequential information of actions.
LG-BPN: Local and Global Blind-Patch Network for Self-Supervised Real-World Denoising
Zichun Wang (Beijing Institute of Technology), Yulun Zhang (ETH Zurich)
RestorationTransformerImage
🎯 What it does: A self-supervised real-world image denoising framework LG-BPN is proposed to remove real noise without clean paired data.
LiDAR-in-the-Loop Hyperparameter Optimization
Félix Goudreault (Algolux), Felix Heide (Princeton University)
Object DetectionDepth EstimationAutonomous DrivingOptimizationHyperparameter SearchPoint Cloud
🎯 What it does: By performing zero-order black-box multi-objective optimization on sensor waveforms and DSP parameters on both simulated and real LiDAR devices, the LiDAR settings are automatically adjusted to enhance 3D object detection and depth estimation performance.
LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation
Song Wang (Zhejiang University), Jianke Zhu (Zhejiang University)
SegmentationAutonomous DrivingKnowledge DistillationMultimodalityPoint Cloud
🎯 What it does: This study proposes a method called LiDAR2Map for online construction of high-precision semantic maps using LiDAR, and enhances the semantic expression capability of LiDAR features through online camera information distillation.
LidarGait: Benchmarking 3D Gait Recognition With Point Clouds
Chuanfu Shen (University of Hong Kong), Shiqi Yu (Southern University of Science and Technology)
RecognitionConvolutional Neural NetworkMultimodalityPoint CloudBenchmark
🎯 What it does: A LiDAR-based gait recognition framework called LidarGait is proposed, and the first large-scale multimodal LiDAR gait benchmark dataset SUSTech1K is constructed.
Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Field
Leheng Li (Hong Kong University of Science and Technology), Ying-Cong Chen (Hong Kong University of Science and Technology)
Object DetectionGenerationData SynthesisAutonomous DrivingNeural Radiance FieldGenerative Adversarial NetworkImage
🎯 What it does: By embedding the pre-trained 2D StyleGAN2 into a 3D NeRF, high-resolution synthetic training samples with precise 3D bounding boxes are generated.
Light Source Separation and Intrinsic Image Decomposition Under AC Illumination
Yusaku Yoshida (Kyushu Institute of Technology), Takahiro Okabe (Kyushu Institute of Technology)
RestorationImage
🎯 What it does: Achieving light source separation and intrinsic image decomposition under alternating illumination through blind source separation and a dichromatic reflection model.
LightedDepth: Video Depth Estimation in Light of Limited Inference View Angles
Shengjie Zhu (Michigan State University), Xiaoming Liu (Michigan State University)
Pose EstimationDepth EstimationAutonomous DrivingOptical FlowVideo
🎯 What it does: A two-step video depth estimation framework is proposed, which first normalizes camera poses using optical flow and monocular depth estimation, and then obtains absolute depth through scale alignment and residual depth learning.
LightPainter: Interactive Portrait Relighting With Freehand Scribble
Yiqun Mei (Johns Hopkins University), Vishal M. Patel (Adobe Inc.)
Image TranslationRestorationConvolutional Neural NetworkImage
🎯 What it does: A portrait relighting interactive system called LightPainter based on freehand doodling is proposed.
LINe: Out-of-Distribution Detection by Leveraging Important Neurons
Yong Hyun Ahn, Seong Tae Kim
Anomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: This paper explores a specific problem in the field of computer vision and proposes a new solution.
LinK: Linear Kernel for LiDAR-Based 3D Perception
Tao Lu (Nanjing University), Limin Wang (Nanjing University)
Object DetectionSegmentationAutonomous DrivingComputational EfficiencyPoint Cloud
🎯 What it does: A LinK module based on a linear kernel generator is proposed for efficient computation of large convolution kernels in LiDAR 3D perception.
Linking Garment With Person via Semantically Associated Landmarks for Virtual Try-On
Keyu Yan (Alibaba Group), Chengjun Xie (Hefei Institute of Physical Science Chinese Academy of Sciences)
Image TranslationGenerationGenerative Adversarial NetworkOptical FlowImage
🎯 What it does: A virtual try-on method that links clothing to the human body through semantically associated landmark points (SAL-VTON) is proposed, significantly reducing alignment errors.
LipFormer: High-Fidelity and Generalizable Talking Face Generation With a Pre-Learned Facial Codebook
Jiayu Wang (Alibaba Group), Jingren Zhou (Alibaba Group)
GenerationData SynthesisTransformerGenerative Adversarial NetworkImageVideoAudio
🎯 What it does: This paper proposes LipFormer, a method that utilizes a pre-trained high-quality facial codebook combined with Transformer to generate high-fidelity, generalizable speaker videos.
Listening Human Behavior: 3D Human Pose Estimation With Acoustic Signals
Yuto Shibata (Keio University), Yoshimitsu Aoki (Keio University)
Pose EstimationConvolutional Neural NetworkGenerative Adversarial NetworkAudio
🎯 What it does: Using low-level acoustic signals actively emitted by a pair of microphones and speakers to infer the three-dimensional posture of the human body.
Lite DETR: An Interleaved Multi-Scale Encoder for Efficient DETR
Feng Li (Hong Kong University of Science and Technology), Lionel M. Ni (Hong Kong University of Science and Technology)
Object DetectionComputational EfficiencyTransformerImage
🎯 What it does: Lite DETR is proposed, a pluggable and efficient Transformer encoder that updates high and low-level feature interactions, significantly reducing the computational load of multi-scale features.
Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation
Ning Zhang (University of Twente), Norman Kerle (University of Twente)
Depth EstimationAutonomous DrivingConvolutional Neural NetworkTransformerImage
🎯 What it does: A lightweight CNN-Transformer hybrid architecture called Lite-Mono is proposed for self-supervised monocular depth estimation. The model is small in size and low in parameter count, yet achieves competitive accuracy.
Local 3D Editing via 3D Distillation of CLIP Knowledge
Junha Hyung (KAIST AI), Jaegul Choo (KAIST AI)
GenerationKnowledge DistillationNeural Radiance FieldGenerative Adversarial NetworkContrastive LearningImagePoint Cloud
🎯 What it does: Achieving local 3D editing of NeRF using text prompts, the method implements local changes to target attributes by generating 3D masks and fusing them at the feature level.