CVPR 2024 Papers — Page 21
IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2716 papers
Relaxed Contrastive Learning for Federated Learning
Seonguk Seo (Seoul National University), Bohyung Han (Seoul National University)
ClassificationFederated LearningContrastive LearningImage
🎯 What it does: A relaxed contrastive learning framework FedRCL is proposed to address the gradient bias and feature representation collapse issues caused by data heterogeneity in federated learning.
RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method
Ming Yan (Xiamen University), Cheng Wang (Xiamen University)
Pose EstimationRecurrent Neural NetworkVideoMultimodalityPoint CloudBenchmark
🎯 What it does: A high-quality multimodal human action dataset RELI11D has been constructed, which includes four types of sensors: RGB, LiDAR, IMU, and event cameras, and a baseline for multimodal human pose estimation called LEIR has been proposed.
Relightable and Animatable Neural Avatar from Sparse-View Video
Zhen Xu (Zhejiang University), Xiaowei Zhou (Zhejiang University)
GenerationOptimizationNeural Radiance FieldVideo
🎯 What it does: Constructing a re-lightable and animatable neural human avatar from a sparse perspective or monocular video under unknown lighting.
Relightable Gaussian Codec Avatars
Shunsuke Saito (Codec Avatars Lab Meta), Giljoo Nam (Codec Avatars Lab Meta)
Auto EncoderGaussian SplattingVideo
🎯 What it does: A real-time light control head avatar model based on 3D Gaussian mixtures is proposed;
Relightful Harmonization: Lighting-aware Portrait Background Replacement
Mengwei Ren (Adobe), He Zhang (Adobe)
Image TranslationImage HarmonizationData SynthesisDiffusion modelImage
🎯 What it does: A lighting-aware diffusion model has been developed for harmonizing lighting and color between portraits and arbitrary backgrounds.
RepAn: Enhanced Annealing through Re-parameterization
Xiang Fei (Xiamen University), Liujuan Cao (DeepWisdom Inc.)
OptimizationKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a training framework called RepAn that combines reparameterization (Rep) with simulated annealing. In each iteration, it first compresses a multi-branch network into a single branch, then expands new parallel branches for learning, thereby achieving the effects of incremental learning and ensemble learning.
RepKPU: Point Cloud Upsampling with Kernel Point Representation and Deformation
Yi Rong (Nanjing University), Tong Lu (Nanjing University)
GenerationData SynthesisTransformerPoint Cloud
🎯 What it does: A novel point cloud upsampling network RepKPU based on kernel point representation and kernel-to-displacement generation is proposed.
Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability Composability and Decomposability from Anatomy via Self Supervision
Mohammad Reza Hosseinzadeh Taher (Arizona State University), Jianming Liang (Arizona State University)
ClassificationRepresentation LearningConvolutional Neural NetworkTransformerContrastive LearningImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper presents Adam-v2, a self-supervised learning framework that learns embedding representations from unlabeled medical images using an anatomical hierarchical structure, which can be directly applied to zero-shot, few-shot, and full fine-tuning tasks.
Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation
Bingxin Ke (ETH Zurich), Konrad Schindler (ETH Zurich)
GenerationDepth EstimationSupervised Fine-TuningDiffusion modelImage
🎯 What it does: By performing lightweight fine-tuning on Stable Diffusion, it is transformed into a monocular depth estimator called Marigold;
RepViT: Revisiting Mobile CNN From ViT Perspective
Ao Wang (Tsinghua University), Guiguang Ding (University of Sheffield)
ClassificationObject DetectionSegmentationConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: A pure lightweight CNN named RepViT is proposed, which draws on the MetaFormer structure of Vision Transformer and other lightweight ViT designs, achieving lower latency and higher accuracy through structural re-parameterization.
Residual Denoising Diffusion Models
Jiawei Liu (Chinese Academy of Sciences), Liangqiong Qu (University of Hong Kong)
RestorationGenerationDiffusion modelImage
🎯 What it does: A Residual Denoising Diffusion Model (RDDM) is proposed, which splits the traditional single denoising diffusion process into two subprocesses: residual diffusion and noise diffusion, achieving unification and interpretability of generation and recovery tasks.
Residual Learning in Diffusion Models
Junyu Zhang (Central South University), Chang Xu (University of Sydney)
GenerationKnowledge DistillationDiffusion modelScore-based ModelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: The paper proposes a residual learning framework that corrects the sampling trajectory of diffusion models by training a correction function, thereby compensating for sampling gaps caused by score estimation errors and discretization errors, and improving the quality of generated images.
Resolution Limit of Single-Photon LiDAR
Stanley H. Chan (Purdue University), Robert K. Henderson (University of Edinburgh)
Depth EstimationGaussian SplattingSimultaneous Localization and MappingPoint Cloud
🎯 What it does: This paper studies the error limits of single-photon LiDAR as the number of pixels changes, deriving a closed-form expression for MSE and validating it through simulations and real data.
Resource-Efficient Transformer Pruning for Finetuning of Large Models
Fatih Ilhan (Georgia Institute of Technology), Ling Liu (Georgia Institute of Technology)
SegmentationOptimizationComputational EfficiencyTransformerSupervised Fine-TuningImageText
🎯 What it does: The RECAP framework is proposed, which adopts an iterative pruning-finetune-update three-stage cycle, loading and training only sub-networks on the GPU, thereby achieving memory-efficient fine-tuning of large-scale Transformers.
Restoration by Generation with Constrained Priors
Zheng Ding (University of California San Diego), Zhihao Xia (Adobe)
RestorationGenerationSupervised Fine-TuningDiffusion modelImage
🎯 What it does: This paper proposes a method for blind image restoration by directly denoising degraded images using a pre-trained diffusion model after injecting sufficient noise, and constraining the generation space through fine-tuning or generating personal albums.
Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning
Dipam Goswami (Universitat Autonoma de Barcelona), Joost van de Weijer (Universitat Autonoma de Barcelona)
ClassificationKnowledge DistillationAdversarial AttackConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: This study investigates exemplar-free continual learning scenarios and proposes an Adversarial Drift Compensation (ADC) method that utilizes adversarial samples to estimate and compensate for the feature drift of old classes, aiming to enhance class-incremental learning performance under small initial tasks.
Rethinking Boundary Discontinuity Problem for Oriented Object Detection
Hang Xu (Hangzhou Dianzi University), Feng Dai (Institute of Computing Technology, Chinese Academy of Sciences)
Object DetectionOptimizationConvolutional Neural NetworkImage
🎯 What it does: A dual optimization framework (Dual-Optimization) and complex exponential encoding (ACM) are proposed to address the boundary discontinuity problem in inclined object detection.
Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolution
Guangyuan Li (Zhejiang University), Lei Zhao (Zhejiang University)
RestorationSuper ResolutionTransformerDiffusion modelImageMagnetic Resonance Imaging
🎯 What it does: A multi-contrast MRI super-resolution method called DiffMSR, which combines a latent diffusion model with a Prior-Guide Large Window Transformer, is designed to generate high-quality, distortion-free super-resolved images in just 4 inference steps.
Rethinking Few-shot 3D Point Cloud Semantic Segmentation
Zhaochong An (ETH Zurich), Serge Belongie (University of Copenhagen)
SegmentationTransformerPoint CloudBenchmark
🎯 What it does: This paper proposes a new few-shot 3D point cloud semantic segmentation method called COSeg, which addresses the performance distortion issues caused by the leakage of point density in the pre-sample and sparse sampling.
Rethinking FID: Towards a Better Evaluation Metric for Image Generation
Sadeep Jayasumana (Google Research), Sanjiv Kumar (Google Research)
GenerationData SynthesisTransformerDiffusion modelContrastive LearningImage
🎯 What it does: A systematic evaluation of the existing image generation evaluation metric FID is conducted, pointing out its assumption failures, low sample efficiency, and inconsistency with human subjective assessments, and a new metric CMMD based on CLIP embeddings and MMD is proposed.
Rethinking Generalizable Face Anti-spoofing via Hierarchical Prototype-guided Distribution Refinement in Hyperbolic Space
Chengyang Hu (Shanghai Jiao Tong University), Lizhuang Ma (Shanghai Jiao Tong University)
RecognitionDomain AdaptationContrastive LearningImage
🎯 What it does: A hierarchical prototype-guided distribution refinement (HPDR) framework based on hyperbolic space is proposed, which uses multi-level prototypes to hierarchically model facial anti-spoofing features and achieve domain generalization.
Rethinking Human Motion Prediction with Symplectic Integral
Haipeng Chen (Jilin University), Yingda Lyu
GenerationPose EstimationGenerative Adversarial NetworkTime SeriesSequential
🎯 What it does: A deep framework called SINN based on symplectic integration is proposed, which uses symplectic representation to provide numerically stable representations of human motion and achieves long-term prediction through a symplectic temporal aggregation module.
Rethinking Inductive Biases for Surface Normal Estimation
Gwangbin Bae (Imperial College London), Andrew J. Davison (Imperial College London)
Depth EstimationConvolutional Neural NetworkRecurrent Neural NetworkImage
🎯 What it does: This paper proposes a method for estimating surface normals from a single RGB image using ray direction and relative rotation learning of adjacent pixel normals.
Rethinking Interactive Image Segmentation with Low Latency High Quality and Diverse Prompts
Qin Liu (University of North Carolina at Chapel Hill), Marc Niethammer (University of North Carolina at Chapel Hill)
SegmentationTransformerSupervised Fine-TuningImageVideoBiomedical Data
🎯 What it does: A novel interactive image segmentation framework called SegNext is proposed, which achieves low latency, high quality, and supports various interactive prompts through dense prompt representation and fusion.
Rethinking Multi-domain Generalization with A General Learning Objective
Zhaorui Tan (Xi'an Jiaotong-Liverpool University), Kaizhu Huang (Duke Kunshan University)
ClassificationSegmentationDepth EstimationDomain AdaptationAuto EncoderImage
🎯 What it does: A general multi-domain generalization target GMDG is proposed, which relaxes the static assumption of target distribution using a learnable Y mapping and decomposes the target into four complementary sub-goals.
Rethinking Multi-view Representation Learning via Distilled Disentangling
Guanzhou Ke (Beijing Jiaotong University), Shengfeng He (Singapore Management University)
Representation LearningAuto EncoderImage
🎯 What it does: A multi-view representation learning framework called MRDD is proposed, which first learns view-consistent representations through Masked Cross-View Prediction, and then uses Distilled Disentangling to eliminate redundancy between consistency and view-specificity, resulting in high-quality view-consistent and specific representations.
Rethinking Prior Information Generation with CLIP for Few-Shot Segmentation
Jin Wang (China University of Petroleum), Weifeng Liu (China University of Petroleum)
SegmentationContrastive LearningImage
🎯 What it does: This paper proposes a method for generating CLIP-based prior information without additional training, combining visual-text alignment and visual-visual matching to achieve fine localization and global guidance for few-shot semantic segmentation.
Rethinking the Evaluation Protocol of Domain Generalization
Han Yu (Tsinghua University), Peng Cui (Tsinghua University)
Domain AdaptationConvolutional Neural NetworkTransformerContrastive LearningImageBenchmark
🎯 What it does: This paper examines the domain generalization evaluation protocol, pointing out that supervised pre-training and oracle model selection can leak test information. It proposes a new protocol that uses self-supervised pre-training or training from scratch, along with multi-test domain evaluation, and re-releases the leaderboard.
Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesis
Yuchao Gu (National University of Singapore), Mike Zheng Shou (National University of Singapore)
GenerationData SynthesisTransformerGenerative Adversarial NetworkImage
🎯 What it does: A new VQ tokenizer SeQ-GAN is proposed, which employs a two-stage training to balance semantic compression and detail preservation, and constructs a visualization process to evaluate the impact of different VQ tokenizers on generative Transformers.
Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data
Xinting Liao (Zhejiang University), Yanchao Tan (Zhejiang University)
Federated LearningRepresentation LearningContrastive LearningImage
🎯 What it does: Proposes the FedU2 framework to address the issues of representation collapse and inconsistency caused by unlabeled data and uneven data distribution in Federated Unsupervised Learning (FUSL).
Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance
Dazhong Shen (Shanghai Artificial Intelligence Laboratory), Yu Liu (Shanghai Artificial Intelligence Laboratory)
SegmentationGenerationDiffusion modelImage
🎯 What it does: Proposes a method for adaptively adjusting the Classifier-Free Guidance (CFG) scale for different semantic regions in text-to-image diffusion models;
Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection
Chuangchuang Tan (Beijing Jiaotong University), Yunchao Wei (University of Nevada)
ClassificationGenerationConvolutional Neural NetworkDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: A local pixel relationship feature based on a generative model upsampling layer (Neighboring Pixel Relationships, NPR) is proposed for generalizing deep fake detection;
Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery
Mubashir Noman (Mohamed bin Zayed University of AI), Fahad Shahbaz Khan (Linköping University)
ClassificationRepresentation LearningTransformerAuto EncoderImage
🎯 What it does: A multi-scale self-supervised pre-training framework called SatMAE++ is proposed, which utilizes masked autoencoders and convolutional upsampling blocks to reconstruct satellite images at multiple scales, thereby enhancing the feature representation of multispectral and optical remote sensing images.
Retraining-Free Model Quantization via One-Shot Weight-Coupling Learning
Chen Tang (Tsinghua University), Wenwu Zhu (Tsinghua University)
CompressionOptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a mixed-precision quantization framework for one-shot training and search, which achieves joint optimization of multiple bit widths by sharing weights during the training phase, and obtains the best bit width configuration through greedy search during the search phase without the need for additional retraining.
Retrieval-Augmented Egocentric Video Captioning
Jilan Xu (Fudan University), Weidi Xie (Shanghai Jiao Tong University)
GenerationRetrievalTransformerContrastive LearningVideoTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes EgoInstructor, a first-person video subtitle generation model enhanced by cross-view retrieval.
Retrieval-Augmented Embodied Agents
Yichen Zhu (Midea Group), Jian Tang (Midea Group)
RetrievalRobotic IntelligenceTransformerMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposes a Retrieval-Augmented Embodied Agent (RAEA) that enhances the learning and execution capabilities of robots in few-shot scenarios through an external policy memory bank.
Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation
Daichi Horita (University of Tokyo), Kiyoharu Aizawa (University of Tokyo)
GenerationRetrievalTransformerImageRetrieval-Augmented Generation
🎯 What it does: This paper proposes a Retrieval-Augmented Layout Transformer (RALF) that can automatically generate multi-element graphic layouts that match the content based on a given image.
Retrieval-Augmented Open-Vocabulary Object Detection
Jooyeon Kim (Korea University), Hyunwoo J. Kim (Korea University)
Object DetectionTransformerLarge Language ModelVision Language ModelImageRetrieval-Augmented Generation
🎯 What it does: This paper proposes a framework called RALF that enhances open vocabulary object detection models by retrieving negative class vocabulary and utilizing 'spoken concepts' generated by large language models.
Revamping Federated Learning Security from a Defender's Perspective: A Unified Defense with Homomorphic Encrypted Data Space
K Naveen Kumar (Indian Institute of Technology Hyderabad), C Krishna Mohan (Indian Institute of Technology Hyderabad)
Federated LearningSafty and PrivacyKnowledge DistillationAdversarial AttackImage
🎯 What it does: This paper proposes a unified Federated Learning defense framework called FCD, which utilizes row permutation homomorphic encryption and KL distillation loss to simultaneously defend against utility evasion attacks and model inversion attacks in the encrypted data space.
Revisiting Adversarial Training at Scale
Zeyu Wang (University of California Santa Cruz), Cihang Xie (University of California Santa Cruz)
OptimizationAdversarial AttackTransformerContrastive LearningImage
🎯 What it does: Implement adversarial training on large-scale models and massive datasets.
Revisiting Adversarial Training Under Long-Tailed Distributions
Xinli Yue (Wuhan University), Lingchen Zhao (Wuhan University)
ClassificationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This study investigates the effects of adversarial training under long-tail distribution, finding that Balanced Softmax Loss is key to RoBal, and proposes AT-BSL, which combines various data augmentation methods to alleviate robust overfitting and significantly enhance robustness.
Revisiting Counterfactual Problems in Referring Expression Comprehension
Zhihan Yu (Beijing University of Posts and Telecommunications), Ruifan Li (Beijing University of Posts and Telecommunications)
RecognitionObject DetectionGenerationRecurrent Neural NetworkContrastive LearningTextMultimodality
🎯 What it does: To address the 'counterfactual' problem in visual-language tasks, the authors propose a counterfactual expression generation method based on fine-grained attributes (CSG) and an end-to-end model (C-REC) that simultaneously performs object localization and counterfactual determination.
Revisiting Global Translation Estimation with Feature Tracks
Peilin Tao (Institute of Automation, Chinese Academy of Sciences), Shuhan Shen (Institute of Automation, Chinese Academy of Sciences)
Pose EstimationAutonomous DrivingSimultaneous Localization and MappingImage
🎯 What it does: A hybrid explicit global translation estimation framework HETA is proposed, which integrates relative translation and feature trajectories to simultaneously estimate camera position and 3D points.
Revisiting Non-Autoregressive Transformers for Efficient Image Synthesis
Zanlin Ni (Tsinghua University), Gao Huang (Tsinghua University)
GenerationData SynthesisOptimizationTransformerDiffusion modelImage
🎯 What it does: This paper proposes AutoNAT, an automatic optimization method for non-autoregressive Transformers (NAT) in image synthesis, by redesigning training and inference strategies.
Revisiting Sampson Approximations for Geometric Estimation Problems
Felix Rydell (KTH Royal Institute of Technology), Viktor Larsson (Lund University)
OptimizationImage
🎯 What it does: This paper provides a theoretical analysis of the Sampson approximation error, presenting a compact upper bound on the error for geometric estimation problems, and validates its effectiveness in various practical visual tasks.
Revisiting Single Image Reflection Removal In the Wild
Yurui Zhu (University of Science and Technology of China), Bo Li (vivo Mobile Communication Co., Ltd)
RestorationConvolutional Neural NetworkImageVideo
🎯 What it does: A video-based pipeline for collecting real reflective images is proposed, and a large-scale high-quality reflective pair dataset RRW is constructed based on this pipeline. Additionally, a maximum reflection filter MaxRF and a two-stage detection-removal network are designed.
Revisiting Spatial-Frequency Information Integration from a Hierarchical Perspective for Panchromatic and Multi-Spectral Image Fusion
Jiangtong Tan (University of Science and Technology of China), Feng Zhao (University of Science and Technology of China)
Image TranslationRestorationSuper ResolutionConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A hierarchical frequency domain fusion network (HFIN) is proposed for full-resolution multispectral fusion (pansharpening) of remote sensing images.
Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer
Wenqiao Zhang (Zhejiang University), Siliang Tang
ClassificationDomain AdaptationImage
🎯 What it does: This paper proposes a Multi-source Active Domain Adaptation (MADA) framework that can transfer multi-source domain knowledge to the target domain and improve classification performance with only a limited number of labeled target samples.
Rewrite the Stars
Xu Ma (Northeastern University), Yun Fu (Northeastern University)
ClassificationComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: This paper theoretically analyzes and empirically proves that element-wise multiplication (star operation) can implicitly map low-dimensional features to high-dimensional nonlinear spaces, and based on this, designs a minimalist and efficient network called StarNet for ImageNet image classification.
RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos
Hongchi Xia (Shanghai Jiao Tong University), Xiaolong Wang (University of California San Diego)
Object DetectionSegmentationPose EstimationNeural Radiance FieldSimultaneous Localization and MappingVideoPoint CloudBenchmark
🎯 What it does: This paper presents and releases WildRGB-D—a panoramic dataset containing approximately 8,500 objects, nearly 20,000 segments of RGB-D videos, and 46 everyday categories, along with object masks, real-scale camera poses, and aggregated point clouds;
Rich Human Feedback for Text-to-Image Generation
Youwei Liang (University of California), Vidhya Navalpakkam (Google)
GenerationTransformerVision Language ModelImageMultimodality
🎯 What it does: Collected rich human feedback on 18K generated images (including suspicious areas, mismatched words, and four-dimensional scores) and trained a multimodal Transformer to predict this feedback for enhancing text-to-image generation models.
RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D
Lingteng Qiu (Chinese University of Hong Kong, Shenzhen), Xiaoguang Han (Chinese University of Hong Kong, Shenzhen)
GenerationData SynthesisDepth EstimationDiffusion modelScore-based ModelPoint CloudMesh
🎯 What it does: Proposed and trained a normal-depth based diffusion model and a chromatic diffusion model to improve geometry and appearance generation from text to 3D.
Riemannian Multinomial Logistics Regression for SPD Neural Networks
Ziheng Chen (University of Trento), Nicu Sebe (University of Trento)
ClassificationOptimizationTime Series
🎯 What it does: A classification layer for SPD networks based on Riemannian Polynomial Logistic Regression (RMLR) is proposed.
RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation
Zeyuan Yang (Tsinghua University), Chuang Gan (UMass Amherst)
Robotic IntelligenceTransformerLarge Language ModelAgentic AIMultimodalityAudio
🎯 What it does: This paper presents RILA, a zero-shot semantic audio-visual navigation (SAVN) agent that utilizes large language models (LLMs) combined with multimodal perception to achieve active exploration and target localization.
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
Tianyu Yu (Tsinghua University), Tat-Seng Chua
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningVision Language ModelTextMultimodality
🎯 What it does: The RLHF-V framework is proposed, which aligns the behavior of multimodal large language models (MLLMs) through fine-grained corrective human feedback, significantly enhancing the model's credibility and reducing hallucinations.
RMem: Restricted Memory Banks Improve Video Object Segmentation
Junbao Zhou (University of Illinois Urbana Champaign), Yu-Xiong Wang (University of Illinois Urbana Champaign)
Object DetectionSegmentationTransformerVideo
🎯 What it does: This paper proposes a concise VOS method called RMem, which enhances the accuracy of video object segmentation by limiting the memory pool size and implementing UCB-friendly memory updates and temporal position encoding.
RMT: Retentive Networks Meet Vision Transformers
Qihang Fan (Institute of Automation, Chinese Academy of Sciences), Ran He (University of Science and Technology Beijing)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: Proposes the RMT network, which integrates explicit spatial priors into the Vision Transformer based on Manhattan Self-Attention and decomposed attention, achieving global modeling with linear complexity;
RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstruction
Baptiste Brument (IRIT, UMR CNRS 5505), Lilian Calvet (OR-X)
Depth EstimationOptimizationNeural Radiance FieldImageBenchmark
🎯 What it does: This paper proposes a neural volume rendering method for 3D reconstruction using multi-view reflectance and normal maps, suitable for multi-view photometric stereo (MVPS).
Robust Depth Enhancement via Polarization Prompt Fusion Tuning
Kei Ikemura (KTH Royal Institute of Technology), Chenyang Lei (Princeton University)
RestorationDepth EstimationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a general framework for enhancing depth maps measured by various depth sensors using polarization image information.
Robust Distillation via Untargeted and Targeted Intermediate Adversarial Samples
Junhao Dong (Nanyang Technological University), Yew-Soon Ong (Nanyang Technological University)
Knowledge DistillationAdversarial AttackTransformerGenerative Adversarial NetworkImage
🎯 What it does: A robust knowledge distillation method called DARWIN is proposed, which is based on intermediate adversarial samples and dual-branch (target-free and target-based) adversarial paths;
Robust Emotion Recognition in Context Debiasing
Dingkang Yang (Fudan University), Lihua Zhang (Fudan University)
RecognitionContrastive LearningImageVideo
🎯 What it does: This paper proposes a context debiasing framework CLEF based on causal counterfactual reasoning to improve the robustness of Context-Aware Emotion Recognition (CAER).
Robust Image Denoising through Adversarial Frequency Mixup
Donghun Ryou (Seoul National University), Bohyung Han (Seoul National University)
RestorationConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A training framework based on adversarial frequency domain mixing (AFM) is proposed to enhance the robustness of image denoising networks against unknown real noise distributions.
Robust Noisy Correspondence Learning with Equivariant Similarity Consistency
Yuchen Yang (Xidian University), Cheng Deng (Xidian University)
RetrievalContrastive LearningMultimodality
🎯 What it does: A regularization method based on Equivariant Similarity Consistency (ESC) is proposed to robustly handle noisy correspondences in image-text matching tasks.
Robust Overfitting Does Matter: Test-Time Adversarial Purification With FGSM
Linyu Tang (Chongqing University), Lei Zhang (Chongqing University)
OptimizationAdversarial AttackImage
🎯 What it does: This paper proposes a pixel-level adversarial purification (TPAP) method for testing on FGSM robust overfitting networks, achieving defense against unknown attacks.
Robust Self-calibration of Focal Lengths from the Fundamental Matrix
Viktor Kocur (Comenius University), Zuzana Kukelova (Czech Technical University in Prague)
Pose EstimationOptimizationSimultaneous Localization and MappingImage
🎯 What it does: An efficient iterative self-calibration method based on the Kruppa equation is proposed, with a simple check for the generation of virtual focal length models added to RANSAC;
Robust Synthetic-to-Real Transfer for Stereo Matching
Jiawei Zhang (Beihang University), Xiao Bai (Beihang University)
Depth EstimationDomain AdaptationAutonomous DrivingKnowledge DistillationImage
🎯 What it does: The study fine-tunes a pre-trained stereo matching network on real-world data while maintaining its generalization ability on unseen domains, and proposes a dynamic filtering and fusion framework (DKT) based on EMA teachers to achieve this goal.
RobustSAM: Segment Anything Robustly on Degraded Images
Wei-Ting Chen (National Taiwan University), Jian Wang (Snap Inc.)
RestorationSegmentationTransformerImage
🎯 What it does: Proposes RobustSAM, which enhances the zero-shot segmentation performance of the Segment Anything Model on low-quality images.
RoDLA: Benchmarking the Robustness of Document Layout Analysis Models
Yufan Chen (Karlsruhe Institute of Technology), Rainer Stiefelhagen (Karlsruhe Institute of Technology)
Object DetectionSegmentationTransformerImageBenchmark
🎯 What it does: A robustness benchmark for document layout analysis models (RoDLA benchmark) is proposed, along with new robustness evaluation metrics and robust models.
RoHM: Robust Human Motion Reconstruction via Diffusion
Siwei Zhang (ETH Zurich), Federica Bogo (Meta Reality Labs Research)
GenerationPose EstimationTransformerDiffusion modelVideo
🎯 What it does: A robust 3D human motion reconstruction framework called RoHM based on diffusion models has been developed, capable of recovering complete and smooth full-body movements from monocular RGB/RGB-D videos with noise and occlusion.
Rolling Shutter Correction with Intermediate Distortion Flow Estimation
Mingdeng Cao (University of Tokyo), Yinqiang Zheng (Tsinghua University)
Image TranslationRestorationConvolutional Neural NetworkOptical FlowImageVideo
🎯 What it does: This paper proposes a framework for directly estimating the intermediate distortion flow from rolling shutter (RS) images to global shutter (GS) images, and recovers GS images through reverse warping.
RoMa: Robust Dense Feature Matching
Johan Edstedt (Linkoping University), Michael Felsberg (Linkoping University)
Convolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: A new dense feature matching model called RoMa is proposed, which uses frozen DINOv2 features for coarse matching and combines them with specialized ConvNet fine features for refinement, achieving robust matching under extreme conditions of scale, illumination, perspective, and texture variation.
Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation
Sihan Liu (Xiamen University), Rongrong Ji (Xiamen University)
SegmentationTransformerImageText
🎯 What it does: A new model for remote sensing image referential segmentation, RMSIN, is proposed, and the largest RRSIS-D dataset is constructed.
Rotation-Agnostic Image Representation Learning for Digital Pathology
Saghir Alfasly (Mayo Clinic), H.R. Tizhoosh (Mayo Clinic)
Representation LearningTransformerContrastive LearningImageBiomedical Data
🎯 What it does: A fast patch selection (FPS) method for digital pathology, a lightweight Transformer model PathDino, and a rotation-invariant self-supervised training framework HistoRotate for pathological images are proposed;
RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation
Peng Lu (Tsinghua Shenzhen International Graduate School), Wenming Yang (Tsinghua Shenzhen International Graduate School)
Object DetectionPose EstimationConvolutional Neural NetworkImage
🎯 What it does: A single-stage real-time multi-person pose estimation framework RTMO has been developed, achieving high accuracy and fast inference by combining coordinate classification with the YOLO architecture.
RTracker: Recoverable Tracking via PN Tree Structured Memory
Yuqing Huang (Harbin Institute of Technology), Ming-Hsuan Yang (University of California Merced)
Object DetectionObject TrackingTransformerVideo
🎯 What it does: A self-recovering visual tracking framework called RTracker is proposed, which utilizes tree-structured memory to achieve dynamic association between the tracker and the detector.
S-DyRF: Reference-Based Stylized Radiance Fields for Dynamic Scenes
Xingyi Li (Huazhong University of Science and Technology), Guosheng Lin (Nanyang Technological University)
Image TranslationGenerationNeural Radiance FieldVideo
🎯 What it does: This work proposes a style transfer method for dynamic 3D radiance fields (S-DyRF) based on reference images, achieving spatial and temporal consistency in style transfer for dynamic 3D scenes.
S2MAE: A Spatial-Spectral Pretraining Foundation Model for Spectral Remote Sensing Data
Xuyang Li (Aerospace Information Research Institute, Chinese Academy of Sciences), Jocelyn Chanussot (University of Grenoble Alpes)
ClassificationObject DetectionTransformerAuto EncoderImage
🎯 What it does: This paper proposes Spatial-Spectral MAE (S2MAE), a 3D mask autoencoder specifically designed for multispectral/hyperspectral remote sensing images.
S2MVTC: a Simple yet Efficient Scalable Multi-View Tensor Clustering
Zhen Long (University of Electronic Science and Technology of China), Ce Zhu (University of Electronic Science and Technology of China)
OptimizationImageVideo
🎯 What it does: This paper proposes a scalable multi-view clustering algorithm based on tensors, S2MVTC, which directly learns the intra/inter-view correlations of embedded features and achieves efficient clustering through anchor graphs and tensor low-frequency approximation.
SaCo Loss: Sample-wise Affinity Consistency for Vision-Language Pre-training
Sitong Wu (Chinese University of Hong Kong), Jiaya Jia (Chinese University of Hong Kong)
RetrievalRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A loss function called SaCo Loss, based on sample-level similarity consistency, is designed to enhance the similarity consistency between different modalities in visual-language pre-training models, and can be used in both zero-shot pre-training and continual pre-training.
SAFDNet: A Simple and Effective Network for Fully Sparse 3D Object Detection
Gang Zhang (Tsinghua University), Xiaolin Hu (Tsinghua University)
Object DetectionAutonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: This paper presents SAFDNet, a fully sparse 3D object detection network.
SAI3D: Segment Any Instance in 3D Scenes
Yingda Yin (Peking University), Baoquan Chen (Peking University)
Object DetectionSegmentationPoint Cloud
🎯 What it does: Proposes SAI3D, a zero-shot 3D instance segmentation method that utilizes geometric priors and 2D segmentations automatically generated by SAM for fusion across multiple views;
Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement
Xiuquan Hou (Xi'an Jiaotong University), Badong Chen (Zhejiang University)
Object DetectionTransformerImage
🎯 What it does: A hierarchical saliency filtering and query refinement DETR framework is proposed, reducing encoding and selection redundancy, significantly improving small object detection performance.
SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation
Jiehong Lin (DexForce Co. Ltd.), Kui Jia (Chinese University of Hong Kong)
Object DetectionSegmentationPose EstimationTransformerImageBenchmark
🎯 What it does: Proposes the SAM-6D framework to achieve zero-shot 6D object pose estimation: first, use SAM to generate candidate segmentations, then filter target instances using semantic, appearance, and geometric matching scores; finally, predict poses using a two-stage point matching network based on background tokens (coarse-fine).
SANeRF-HQ: Segment Anything for NeRF in High Quality
Yichen Liu (Hong Kong University of Science and Technology), Yu-Wing Tai (Dartmouth College)
Object DetectionSegmentationNeural Radiance FieldPoint Cloud
🎯 What it does: Achieving high-quality 3D object segmentation in NeRF scenes using the Segment Anything model.
SAOR: Single-View Articulated Object Reconstruction
Mehmet Aygun (University of Edinburgh), Oisin Mac Aodha (University of Edinburgh)
SegmentationPose EstimationImage
🎯 What it does: Under a single image, a self-supervised model (SAOR) is trained to predict the 3D shape, texture, and camera pose of animals (including birds, quadrupeds, etc.).
Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion
Zuoyue Li (ETH Zurich), Martin R. Oswald (ETH Zurich)
GenerationData SynthesisDiffusion modelImageVideoPoint Cloud
🎯 What it does: Generate 3D urban scenes that can be rendered from any viewpoint directly using satellite images, and output coherent video sequences.
SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation
Aysim Toker (Technical University of Munich), Laura Leal-Taixé (NVIDIA)
SegmentationGenerationData SynthesisDiffusion modelImage
🎯 What it does: Learn the joint distribution p(x,y) and use a diffusion model to synthesize new image-mask pairs to enhance the training set for remote sensing semantic segmentation.
SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes
Yi-Hua Huang (University of Hong Kong), Xiaojuan Qi (University of Hong Kong)
GenerationData SynthesisNeural Radiance FieldGaussian SplattingVideo
🎯 What it does: This paper proposes a method that utilizes sparse control points and time-dependent deformation MLP to drive 3D Gaussian splatting, achieving high-quality new view synthesis and editable motion for monocular dynamic videos.
SC-Tune: Unleashing Self-Consistent Referential Comprehension in Large Vision Language Models
Tongtian Yue (University of Chinese Academy of Sciences), Jing Liu (University of Chinese Academy of Sciences)
RecognitionObject DetectionGenerationTransformerReinforcement LearningVision Language ModelImageMultimodality
🎯 What it does: In large-scale visual language models (LVLM), a self-consistency tuning (SC-Tune) framework is proposed, which enhances the model's self-consistency in object-level reference expression generation (REG) and localization (REC) through cyclic descriptor-locator dual-modal training, improving generalization ability on unseen data.
Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering
Tao Lu (Nanjing University), Bo Dai
GenerationData SynthesisComputational EfficiencyGaussian SplattingPoint Cloud
🎯 What it does: We propose Scaffold-GS, a hierarchical 3D Gaussian rendering framework based on anchor points, which achieves adaptive rendering based on the viewpoint.
Scalable 3D Registration via Truncated Entry-wise Absolute Residuals
Tianyu Huang (Chinese University of Hong Kong), Yun-Hui Liu (University of Pennsylvania)
Autonomous DrivingOptimizationPoint Cloud
🎯 What it does: A robust 3D registration method based on Truncated Element-wise Absolute Residual (TEAR) is proposed, utilizing decomposition and branch-and-bound to achieve global optimality and scalability to tens of millions of point pairs.
Scaled Decoupled Distillation
Shicai Wei (University of Electronic Science and Technology of China), Yang Luo (University of Electronic Science and Technology of China)
ClassificationKnowledge DistillationImage
🎯 What it does: Proposed a logit knowledge distillation method based on multi-scale decoupling (SDD)
Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion
Lucas Nunes (Center for Robotics, University of Bonn), Cyrill Stachniss (Lamarr Institute for Machine Learning and Artificial Intelligence)
RestorationSegmentationGenerationAutonomous DrivingDiffusion modelPoint Cloud
🎯 What it does: Scene completion is achieved by using a point-level denoising diffusion model on a single frame of LiDAR point cloud.
Scaling Laws for Data Filtering-- Data Curation cannot be Compute Agnostic
Sachin Goyal (Carnegie Mellon University), J. Zico Kolter (Carnegie Mellon University)
Data-Centric LearningTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: This study investigates the dynamic balance between data filtering strategies and computational budgets in large-scale visual language model training, and proposes a scaling law that can predict the training effects of mixed data pools of different quality.
Scaling Laws of Synthetic Images for Model Training ... for Now
Lijie Fan (Massachusetts Institute of Technology), Yonglong Tian (Google Research)
ClassificationGenerationData SynthesisDiffusion modelImage
🎯 What it does: This study investigates the scaling laws of generating synthetic images from text-to-image models and compares their effectiveness in supervised classification and CLIP training.
Scaling Up Dynamic Human-Scene Interaction Modeling
Nan Jiang (Peking University), Siyuan Huang (Beijing Institute of Technology)
GenerationData SynthesisTransformerDiffusion modelVideo
🎯 What it does: The largest motion capture HSI dataset, TRUMANS, has been constructed, and a real-time, arbitrary-length HSI generation method based on autoregressive diffusion models has been proposed.
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild
Fanghua Yu (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences), Chao Dong (Shanghai AI Laboratory)
RestorationSuper ResolutionDiffusion modelImageTextMultimodality
🎯 What it does: An image restoration system SUPIR based on a large diffusion model (SDXL) has been constructed, capable of achieving high-quality blind restoration in real scenes and supporting controllable restoration based on text prompts.
Scaling Up Video Summarization Pretraining with Large Language Models
Dawit Mureja Argaw (KAIST), Joon Son Chung (KAIST)
GenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningVideoTextMultimodality
🎯 What it does: By using large language models (LLM) to perform extractive summarization on the transcribed text of long videos, we automatically generate 250K video-summary pairs as pre-training data and propose a Transformer-based autoregressive video summarization model.
ScanFormer: Referring Expression Comprehension by Iteratively Scanning
Wei Su (Zhejiang University), Xi Li (Zhejiang University)
RetrievalTransformerVision Language ModelImage
🎯 What it does: This paper proposes ScanFormer, a visual language Transformer that iteratively removes irrelevant visual regions from coarse to fine for referring expression retrieval.
SCE-MAE: Selective Correspondence Enhancement with Masked Autoencoder for Self-Supervised Landmark Estimation
Kejia Yin (University of Toronto), David B. Lindell (University of Toronto)
RecognitionPose EstimationTransformerAuto EncoderContrastive LearningImage
🎯 What it does: This paper proposes a two-stage self-supervised facial keypoint detection framework SCE-MAE, which first uses a Masked Autoencoder (MAE) for pre-training on images to obtain features with better local discriminability; subsequently, it selectively refines the keypoint correspondences through Corresponding Approximation and Refinement Blocks (CARB) and Local Constraint Rejection Loss (LCR).