ICCV 2023 Papers — Page 17
IEEE/CVF International Conference on Computer Vision · 2156 papers
Robust One-Shot Face Video Re-enactment using Hybrid Latent Spaces of StyleGAN2
Trevine Oorloff (University of Maryland), Yaser Yacoob (University of Maryland)
Image TranslationGenerationGenerative Adversarial NetworkImageVideo
🎯 What it does: A hybrid latent space framework based on StyleGAN2 is proposed, capable of one-shot facial video reproduction at a resolution of 1024×1024, supporting attribute editing while being robust to the pose and expression of the source frame.
Robust Referring Video Object Segmentation with Cyclic Structural Consensus
Xiang Li (Carnegie Mellon University), Yan Lu (Microsoft Research Asia)
Object DetectionSegmentationTransformerVideoTextMultimodality
🎯 What it does: A robust R-VOS task R2-VOS is designed, proposing a cyclic structural consistency constraint, jointly training segmentation and text reconstruction, and constructing a new R2-YouTube-VOS evaluation dataset.
Robustifying Token Attention for Vision Transformers
Yong Guo (Max Planck Institute for Informatics), Bernt Schiele (Max Planck Institute for Informatics)
ClassificationSegmentationDomain AdaptationTransformerImage
🎯 What it does: This paper proposes Token-aware Average Pooling (TAP) and Attention Diversification Loss (ADL), which alleviate the token overfocusing problem and enhance robustness by allowing ViT to focus more on local neighborhoods during the self-attention process and suppress the attention similarity between different tokens.
Role-Aware Interaction Generation from Textual Description
Mikihiro Tanaka (LINE Corporation), Kent Fujiwara (LINE Corporation)
GenerationTransformerDiffusion modelVideoMultimodality
🎯 What it does: This study investigates a model for generating perceived interactive actions between two characters from textual descriptions.
ROME: Robustifying Memory-Efficient NAS via Topology Disentanglement and Gradient Accumulation
Xiaoxing Wang (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
ClassificationNeural Architecture SearchImage
🎯 What it does: A single-path differentiable neural architecture search method named ROME is proposed to address the performance collapse issue in single-path searches.
Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with Monocular Videos
Yikai Wang (Beijing National Research Center for Information Science and Technology), Xiao Yang (Beijing National Research Center for Information Science and Technology)
SegmentationPose EstimationNeural Radiance FieldOptical FlowVideo
🎯 What it does: The study reconstructs a 3D model of non-rigid objects from monocular RGB videos, proposing the Root Pose Decomposition (RPD) method, which constructs a shared canonical space and corrects the root pose by learning dense local transformations.
Rosetta Neurons: Mining the Common Units in a Model Zoo
Amil Dravid, Assaf Shocher
Image TranslationGenerationData SynthesisConvolutional Neural NetworkTransformerGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes and implements an unsupervised method to mine shared neural units (referred to as Rosetta Neurons) across various architectures, tasks, and supervision methods in visual models and constructs a dictionary.
RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation
Zhexiong Wan (Northwestern Polytechnical University), Yuchao Dai (Northwestern Polytechnical University)
Autonomous DrivingOptical FlowImageMultimodalityPoint Cloud
🎯 What it does: Joint optical flow and scene flow estimation is achieved by combining RGB images, point clouds, and event cameras.
RPG-Palm: Realistic Pseudo-data Generation for Palmprint Recognition
Lei Shen (Tencent), Wei Jia (Hefei University of Technology)
RecognitionGenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: A new realistic pseudo palmprint generation model (RPG) is proposed to synthesize palmprints with a large number of identities, addressing the issue of the lack of large-scale public datasets in palmprint recognition.
RSFNet: A White-Box Image Retouching Approach using Region-Specific Color Filters
Wenqi Ouyang (Alibaba Group), Xuansong Xie (Alibaba Group)
Image HarmonizationRestorationConvolutional Neural NetworkImage
🎯 What it does: A white-box image beautification framework RSFNet is proposed, utilizing parallel region-specific filters to achieve fine-grained color adjustments and providing editable filter parameters.
s-Adaptive Decoupled Prototype for Few-Shot Object Detection
Jinhao Du (Baidu), Jingdong Wang (Baidu)
Object DetectionConvolutional Neural NetworkImage
🎯 What it does: For few-shot object detection, a σ-adaptive decoupled prototype (σ-ADP) method is proposed to provide high-quality, task-specific prototype vectors in both the RPN and detection head, thereby enhancing detection performance.
S-TREK: Sequential Translation and Rotation Equivariant Keypoints for Local Feature Extraction
Emanuele Santellani (Graz University of Technology), Friedrich Fraundorfer (Sony Europe)
RecognitionObject DetectionConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: This study proposes S-TREK, a deep keypoint detector designed to be invariant to both translation and rotation, equipped with a lightweight descriptor network for end-to-end local feature extraction.
S-VolSDF: Sparse Multi-View Stereo Regularization of Neural Implicit Surfaces
Haoyu Wu (Stony Brook University), Dimitris Samaras (Stony Brook University)
Depth EstimationOptimizationNeural Radiance FieldImage
🎯 What it does: By combining multi-view stereo (MVS) probability volumes with neural implicit surface rendering (VolSDF), we optimize 3D reconstruction and novel view synthesis using correspondence constraints under sparse viewpoints.
S3IM: Stochastic Structural SIMilarity and Its Unreasonable Effectiveness for Neural Fields
Zeke Xie (Baidu Research), Mingming Sun (Baidu Research)
RestorationGenerationNeural Radiance FieldImage
🎯 What it does: A new multi-channel training paradigm S3IM is proposed, which utilizes the structural similarity of randomly sampled pixel groups to train NeRF and its variants as well as neural surface reconstruction models.
SA-BEV: Generating Semantic-Aware Bird's-Eye-View Feature for Multi-view 3D Object Detection
Jinqing Zhang (Beihang University), Yunhong Wang (Beihang University)
Object DetectionAutonomous DrivingPoint Cloud
🎯 What it does: A semantic-aware bird's-eye view feature generation method is proposed to enhance multi-view 3D detection.
SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretability
Wei Huang (Purple Mountain Laboratories), Xiaowei Huang (University of Liverpool)
Explainability and InterpretabilityConvolutional Neural NetworkImage
🎯 What it does: This paper proposes two metrics for evaluating the robustness of interpretability in deep learning models and implements efficient estimation of these metrics based on black-box methods (genetic algorithms and subset simulation).
SAFE: Machine Unlearning With Shard Graphs
Yonatan Dukler (Amazon Web Services AI Labs), Stefano Soatto (Amazon Web Services AI Labs)
ClassificationRecognitionGraph Neural NetworkTransformerImage
🎯 What it does: A collaborative memory elimination method based on fragment graphs, SAFE, is proposed to achieve controllable forgetting and high-precision inference for large models.
SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detection
Samuel Wilson (Queensland University of Technology), Niko Sünderhauf (University of Adelaide)
Object DetectionAnomaly DetectionAutonomous DrivingAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A post-hoc OOD detection method called SAFE is proposed, which utilizes sensitive feature vectors extracted from the residual convolution + BatchNorm layers of a pre-trained detector backbone, and trains an auxiliary MLP to distinguish between ID and OOD detection results.
SAFL-Net: Semantic-Agnostic Feature Learning Network with Auxiliary Plugins for Image Manipulation Detection
Zhihao Sun (Institute of Computing Technology), Juan Cao (Renmin University of China)
Anomaly DetectionConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes SAFL-Net, which achieves semantic-agnostic feature learning for image tampering detection and localization through a semantic suppression module and a boundary guidance module.
SAGA: Spectral Adversarial Geometric Attack on 3D Meshes
Tomer Stolik (Tel Aviv University), Shai Avidan (Tel Aviv University)
Adversarial AttackAuto EncoderMesh
🎯 What it does: A spectral domain-based geometric adversarial attack method (SAGA) is proposed for 3D mesh autoencoders, enabling the model to output a geometric shape nearly identical to the target mesh after the input is subjected to minor perturbations.
SAL-ViT: Towards Latency Efficient Private Inference on ViT using Selective Attention Search with a Learnable Softmax Approximation
Yuke Zhang (University of Southern California), Peter A. Beerel (University of Southern California)
ClassificationSafty and PrivacyComputational EfficiencyTransformerSupervised Fine-TuningImage
🎯 What it does: This paper designs a ViT framework for private inference called SAL-ViT, proposing a learnable second-order polynomial softmax approximation (L2Q) and a selective attention search (SAS) technique to significantly reduce private inference latency and improve accuracy.
SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation
Juil Koo (Korea Advanced Institute of Science and Technology), Minhyuk Sung (Korea Advanced Institute of Science and Technology)
GenerationData SynthesisTransformerDiffusion modelPoint CloudMesh
🎯 What it does: A two-stage diffusion model (SALAD) based on fragmented implicit 3D representation is proposed for high-quality 3D shape generation and part-level editing.
Saliency Regularization for Self-Training with Partial Annotations
Shouwen Wang (Huazhong University of Science and Technology), Zhigang Zeng (Huazhong University of Science and Technology)
ClassificationObject DetectionConvolutional Neural NetworkImage
🎯 What it does: A self-training based multi-label classification framework is proposed, primarily addressing partially labeled data by introducing Saliency Regularization (SR) and Consistency Regularization (CR).
Sample-adaptive Augmentation for Point Cloud Recognition Against Real-world Corruptions
Jie Wang (Beijing Institute of Technology), Jianan Li (Chinese University of Hong Kong)
RecognitionAdversarial AttackTransformerPoint Cloud
🎯 What it does: This paper proposes AdaptPoint, an adaptive point cloud enhancement framework that improves the model's robustness to real-world corruption by generating local deformations and occlusions based on the point cloud structure.
Sample-wise Label Confidence Incorporation for Learning with Noisy Labels
Chanho Ahn (Samsung Advanced Institute of Technology), Seungju Han (Samsung Advanced Institute of Technology)
ClassificationSupervised Fine-TuningImage
🎯 What it does: A training framework based on sample label confidence is proposed for handling image classification tasks with noisy labels.
Sample4Geo: Hard Negative Sampling For Cross-View Geo-Localisation
Fabian Deuser (Institute for Distributed Intelligent Systems University of the Bundeswehr Munich), Norbert Oswald (Institute for Distributed Intelligent Systems University of the Bundeswehr Munich)
RetrievalRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: A cross-view geographic localization method based on a single shared weight ConvNeXt encoder and contrastive learning has been developed, utilizing symmetric InfoNCE loss and two-stage hard negative sampling to improve matching accuracy.
SAMPLING: Scene-adaptive Hierarchical Multiplane Images Representation for Novel View Synthesis from a Single Image
Xiaoyu Zhou (Peking University), Ming-Hsuan Yang (University of California)
GenerationData SynthesisAutonomous DrivingTransformerNeural Radiance FieldImage
🎯 What it does: Achieving perspective synthesis of large-scale outdoor scenes from a single image.
Sat2Density: Faithful Density Learning from Satellite-Ground Image Pairs
Ming Qian (Wuhan University), Nan Xue (Ant Group)
GenerationData SynthesisGenerative Adversarial NetworkImageVideo
🎯 What it does: A satellite-to-ground view synthesis method based on density fields, called Sat2Density, is proposed, which can learn 3D geometry and generate multi-view ground panoramic videos using only a pair of satellite-ground images.
SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding
Favyen Bastani (Allen Institute for AI), Aniruddha Kembhavi (Allen Institute for AI)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: A large-scale multi-task remote sensing dataset SATLASPRETRAIN has been constructed, and a unified model SATLASNET has been proposed, supporting seven different label types; it also demonstrates significant performance improvements of this dataset on multiple downstream remote sensing tasks.
SATR: Zero-Shot Semantic Segmentation of 3D Shapes
Ahmed Abdelreheem (King Abdullah University of Science and Technology), Peter Wonka (King Abdullah University of Science and Technology)
Object DetectionSegmentationPoint CloudMesh
🎯 What it does: Using an open-source zero-shot 2D object detector and topological reweighting techniques, multi-view bounding box predictions are transformed into 3D mesh semantic segmentation.
SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data
Mohammad Zohaib (Italian Institute of Technology), Alessio Del Bue (Italian Institute of Technology)
Object DetectionPose EstimationPoint Cloud
🎯 What it does: This paper proposes a completely unsupervised 3D keypoint detection method called SC3K, which can efficiently and robustly infer semantically consistent keypoints that closely adhere to surfaces in point cloud data under arbitrary rotations, noise, and downsampling.
Scalable Diffusion Models with Transformers
William Peebles (University of California Berkeley), Saining Xie (New York University)
GenerationData SynthesisComputational EfficiencyTransformerDiffusion modelImage
🎯 What it does: This paper explores a novel diffusion model based on the transformer architecture, called Diffusion Transformers (DiTs), which replaces the traditional U-Net as the backbone network for the latent diffusion model of images.
Scalable Multi-Temporal Remote Sensing Change Data Generation via Simulating Stochastic Change Process
Zhuo Zheng (Wuhan University), Yanfei Zhong (Wuhan University)
GenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: A scalable multi-temporal remote sensing change data generation method called Changen is proposed based on the Generative Probability Change Model (GPCM), which can automatically generate controllable change pairs from single-time images and their semantic segmentation images.
Scalable Video Object Segmentation with Simplified Framework
Qiangqiang Wu (City University of Hong Kong), Antoni B. Chan (City University of Hong Kong)
Object DetectionSegmentationTransformerVideo
🎯 What it does: A semi-supervised video object segmentation framework called SimVOS is proposed, which uses a single ViT backbone to achieve a unified process for feature extraction and matching.
Scale-Aware Modulation Meet Transformer
Weifeng Lin (South China University of Technology), Lianwen Jin (South China University of Technology)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: A new visual Transformer backbone network called Scale-Aware Modulation Transformer (SMT) is designed, which combines convolution with Transformer to achieve multi-scale feature fusion and local-to-global dependency modeling.
Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning
Colorado J Reed (Berkeley AI Research), Trevor Darrell (Berkeley AI Research)
ClassificationSegmentationRepresentation LearningTransformerAuto EncoderImage
🎯 What it does: A scale-aware self-supervised pre-training framework called Scale-MAE has been developed for learning representations of multi-scale remote sensing images.
Scaling Data Generation in Vision-and-Language Navigation
Zun Wang (Australian National University), Yu Qiao (OpenGVLab Shanghai AI Laboratory)
GenerationData SynthesisGraph Neural NetworkVision Language ModelGenerative Adversarial NetworkImageTextMultimodality
🎯 What it does: This paper proposes the ScaleVLN large-scale visual and language navigation data generation paradigm, utilizing over 1200 3D scenes from HM3D and Gibson to construct a fully covered, obstacle-free navigation map, restore rendered images, and generate 4.9 million R2R-style instruction-trajectory pairs.
SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval
Sunjae Yoon (Korea Advanced Institute of Science and Technology), Chang D. Yoo (Korea Advanced Institute of Science and Technology)
RetrievalTransformerContrastive LearningVideoTextMultimodality
🎯 What it does: A weakly supervised video moment retrieval network SCANet based on scene complexity is designed, which can adaptively generate and enhance candidate segments to locate the moments corresponding to the queries.
ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes
Chandan Yeshwanth (Technical University of Munich), Angela Dai (Technical University of Munich)
SegmentationData SynthesisConvolutional Neural NetworkNeural Radiance FieldImageVideoMultimodalityPoint CloudBenchmark
🎯 What it does: A large-scale, high-resolution 3D indoor scene dataset called ScanNet++ has been created, which includes laser scans, DSLR images, iPhone RGB-D data, as well as open vocabulary and multi-label semantic annotations, and provides corresponding benchmarks.
Scanning Only Once: An End-to-end Framework for Fast Temporal Grounding in Long Videos
Yulin Pan (Alibaba Group), Deli Zhao (Alibaba Group)
RecognitionRetrievalComputational EfficiencyTransformerVideo
🎯 What it does: This paper proposes SOONet, an end-to-end framework that can complete long video temporal localization in a single execution.
ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering
Andrea Ramazzina (Mercedes-Benz), Felix Heide (Princeton University)
RestorationDepth EstimationAutonomous DrivingNeural Radiance FieldImagePoint Cloud
🎯 What it does: This paper proposes a physics-based inverse neural rendering method called ScatterNeRF, which can separate fog-free backgrounds and scattering media from multi-view foggy images, and reconstruct fog-free views and depth information within the same scene.
Scene as Occupancy
Wenwen Tong (Shanghai AI Laboratory), Hongyang Li (Shanghai AI Laboratory)
Object DetectionSegmentationAutonomous DrivingTransformerSimultaneous Localization and MappingOptical FlowPoint CloudBenchmark
🎯 What it does: Proposes the OccNet framework and the OpenOcc benchmark, utilizing multi-view vision to construct dense 3D occupancy maps, achieving occupancy prediction and supporting multiple tasks.
Scene Graph Contrastive Learning for Embodied Navigation
Kunal Pratap Singh (Allen Institute for AI), Aniruddha Kembhavi (Allen Institute for AI)
Robotic IntelligenceRecurrent Neural NetworkGraph Neural NetworkReinforcement LearningContrastive LearningGraph
🎯 What it does: This paper proposes the use of scene graphs as auxiliary supervision, designing a Scene Graph Contrastive (SGC) loss to train embodied AI agents that can better capture object semantics, relationships, and history.
Scene Matters: Model-based Deep Video Compression
Lv Tang (University of Chinese Academy of Sciences), Xiaoqi Ma (University of Chinese Academy of Sciences)
CompressionTransformerNeural Radiance FieldContrastive LearningOptical FlowVideo
🎯 What it does: A model-based video compression (MVC) framework is proposed based on implicit neural representations (INR), encoding an entire segment of the same scene into network parameters at once, discarding traditional block/frame-level redundancy prediction.
Scene-Aware Feature Matching
Xiaoyong Lu (Southeast University), Songlin Du (Southeast University)
Pose EstimationRetrievalTransformerImage
🎯 What it does: The SAM (Scene-Aware Matching) model is proposed, which achieves multi-level feature matching by introducing a scene-aware 'group token' in point-level features.
Scene-Aware Label Graph Learning for Multi-Label Image Classification
Xuelin Zhu (Southeast University), Jiuxin Cao (Southeast University)
ClassificationGraph Neural NetworkImage
🎯 What it does: A Scene-Aware Label Graph Learning (SALGL) framework is proposed, which detects image scenes in an unsupervised manner and dynamically maintains a label co-occurrence matrix for each scene. Subsequently, it enhances multi-label image classification performance by utilizing scene-aware co-occurrence weights through graph propagation in the label graph.
SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields
Anh-Quan Cao (Inria), Raoul de Charette (Inria)
Data SynthesisDepth EstimationAutonomous DrivingNeural Radiance FieldContrastive LearningImagePoint Cloud
🎯 What it does: A single-view self-supervised 3D scene reconstruction framework based on NeRF, named SceneRF, is proposed. This framework trains solely on RGB image sequences with pose information and can synthesize depth views from arbitrary positions and angles on a single input image, followed by TSDF fusion to obtain a complete 3D model.
Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation
Yuxin Jiang (Nanyang Technological University), Chen Change Loy (Nanyang Technological University)
Image TranslationGenerationGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: This paper presents Scenimefy—a semi-supervised image-to-image translation framework that utilizes semantic constraints to fine-tune StyleGAN for generating pseudo-paired data. It then employs Mask2Former for semantic segmentation filtering, and finally incorporates Patch-wise Contrastive Style Loss in the joint training of supervised and unsupervised branches to achieve translation from real scenes to high-quality anime scenes.
SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gap
Daehee Kim (NAVER Cloud AI), Taeho Kil (Seoul National University)
RecognitionDomain AdaptationTransformerContrastive LearningImageText
🎯 What it does: This paper studies a general text understanding pre-training method called SCOB, which bridges the domain gap between document images and scene text images through character-level supervised contrastive learning and online text rendering.
Score Priors Guided Deep Variational Inference for Unsupervised Real-World Single Image Denoising
Jun Cheng (Huazhong University of Science and Technology), Shan Tan (Huazhong University of Science and Technology)
RestorationScore-based ModelImage
🎯 What it does: A score-based deep variational inference (ScoreDVI) method is proposed for unsupervised real-world denoising of single images.
Score-Based Diffusion Models as Principled Priors for Inverse Imaging
Berthy T. Feng (California Institute of Technology), William T. Freeman (Google Research)
RestorationDiffusion modelScore-based ModelImageOrdinary Differential Equation
🎯 What it does: This paper transforms the score-based diffusion model into a differentiable image prior and uses it for solving Bayesian inverse problems without hyperparameters.
Scratch Each Other's Back: Incomplete Multi-Modal Brain Tumor Segmentation via Category Aware Group Self-Support Learning
Yansheng Qiu (Wuhan University), Zheng Wang (Wuhan University)
SegmentationKnowledge DistillationMultimodalityBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a category-aware group self-support learning framework (GSS) to enhance feature extraction and mutual distillation effects for multimodal brain tumor segmentation under missing modality conditions.
Scratching Visual Transformer's Back with Uniform Attention
Nam Hyeon-Woo (POSTECH), Tae-Hyun Oh (POSTECH)
ClassificationObject DetectionSegmentationTransformerSupervised Fine-TuningImage
🎯 What it does: This paper analyzes the entropy and gradient characteristics of attention maps in Vision Transformer (ViT) and finds that ViT tends to learn dense but hard-to-converge global attention during training. To address this, the authors propose the Context Broadcasting (CB) module, which injects a global average token into each token at the end of the MLP, achieving uniform global attention. This significantly alleviates the burden of MSA through lightweight implementation (no additional parameters, just one line of code), allowing it to focus on sparse, informative interactions.
Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields
Xiangyu Wang (Zhejiang University), Jiming Chen (Zhejiang University)
Data SynthesisOptimizationKnowledge DistillationNeural Radiance FieldImage
🎯 What it does: An interactive pixel-level NeRF editing framework called Seal-3D has been developed, supporting instant preview and various geometry/color editing tools.
Search for or Navigate to? Dual Adaptive Thinking for Object Navigation
Ronghao Dang (Tongji University), Qijun Chen (Tongji University)
Robotic IntelligenceConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: A dual adaptive thinking framework is proposed to achieve dynamic switching between search thinking and navigation thinking, enhancing object navigation performance.
See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data
Yuhang Lu (ShanghaiTech University), Yuexin Ma (ShanghaiTech University)
SegmentationAutonomous DrivingTransformerContrastive LearningMultimodalityPoint Cloud
🎯 What it does: A multi-modal zero-shot point cloud semantic segmentation method is proposed, which utilizes images and point clouds to construct richer visual features.
SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes
Nicolas Larue (CY Cergy Paris University), Vassilis Christophides (University of Ljubljana)
Anomaly DetectionConvolutional Neural NetworkContrastive LearningImageVideo
🎯 What it does: This paper proposes SeeABLE, a deepfake detection framework based on single-class self-supervised anomaly detection, which generates pseudo samples using soft differences (local slight perturbations) and maps them to predefined uniform distribution prototypes to obtain anomaly scores.
Seeing Beyond the Patch: Scale-Adaptive Semantic Segmentation of High-resolution Remote Sensing Imagery based on Reinforcement Learning
Yinhe Liu (Wuhan University), Yanfei Zhong (Wuhan University)
SegmentationConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: To address the issue of sliding window scale limitations in high-resolution remote sensing image segmentation, this paper proposes GeoAgent—a scale-adaptive semantic segmentation framework based on reinforcement learning, which can dynamically select appropriate image block scales to obtain richer contextual information, thereby improving segmentation accuracy.
SEFD: Learning to Distill Complex Pose and Occlusion
ChangHee Yang (Pusan National University), SukJu Kang (Sogang University)
Pose EstimationKnowledge DistillationConvolutional Neural NetworkSupervised Fine-TuningMesh
🎯 What it does: A method called SMPL Edge Feature Distillation (SEFD) is proposed, which uses overlapping edges generated by the SMPL model (containing occlusion information) as a teacher to guide a student network that only uses ordinary edges to complete 3D human mesh estimation in complex poses and occluded scenes.
SegGPT: Towards Segmenting Everything in Context
Xinlong Wang (Beijing Academy of Artificial Intelligence), Tiejun Huang (Peking University)
SegmentationTransformerPrompt EngineeringImage
🎯 What it does: SegGPT proposes a universal model capable of completing various segmentation tasks in a single context reasoning. By employing a randomly colored in-context learning framework during training on different segmentation data, the model relies on context rather than specific colors to accomplish tasks.
Segment Anything
Alexander Kirillov (Meta AI Research), Ross Girshick (Meta AI Research)
Object DetectionSegmentationTransformerImage
🎯 What it does: Proposed a promptable segmentation task, Segment Anything Model (SAM), and the SA-1B dataset, establishing a foundational model for image segmentation.
Segment Every Reference Object in Spatial and Temporal Spaces
Jiannan Wu (University of Hong Kong), Ping Luo (University of Hong Kong)
Object DetectionObject TrackingSegmentationTransformerImageVideoTextMultimodality
🎯 What it does: A unified model called UniRef is proposed, which can accomplish three major tasks: referring image segmentation (RIS), referring video segmentation (RVOS), and video object segmentation (VOS) within a single network framework.
Segmentation of Tubular Structures Using Iterative Training with Tailored Samples
Wei Liao (Independent Researcher)
SegmentationConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: An iterative training framework based on the minimum path is proposed to simultaneously generate segmentation masks and centerlines for tubular structures, enhancing the consistency between training samples and inference through iterative sampling.
Segmenting Known Objects and Unseen Unknowns without Prior Knowledge
Stefano Gasperini (Technical University of Munich), Federico Tombari (Google)
Object DetectionSegmentationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a new panoramic segmentation setting—holistic segmentation—to address the segmentation problem in the presence of unseen unknown categories (unknown objects), and presents the corresponding end-to-end model U3HS, which can simultaneously achieve panoramic segmentation of known categories and instantiation of unknown objects without relying on any prior knowledge of unknown categories.
SegPrompt: Boosting Open-World Segmentation via Category-Level Prompt Learning
Muzhi Zhu (Zhejiang University), Chunhua Shen (Zhejiang University)
Object DetectionSegmentationTransformerPrompt EngineeringImageBenchmark
🎯 What it does: Proposes the SegPrompt mechanism and the LVIS-OW benchmark to achieve better evaluation and improvement of open-world instance segmentation.
SegRCDB: Semantic Segmentation via Formula-Driven Supervised Learning
Risa Shinoda (National Institute of Advanced Industrial Science and Technology), Hirokatsu Kataoka (Tokyo Institute of Technology)
SegmentationTransformerSupervised Fine-TuningImage
🎯 What it does: A semantic segmentation pre-training dataset SegRCDB based on Formula-Driven Supervised Learning (FDSL) is proposed and implemented, utilizing radial contours to generate precise pixel-level masks, completing pre-training without real images.
SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage
Song Park (NAVER AI Lab), Sangdoo Yun (NAVER AI Lab)
CompressionOptimizationTransformerGenerative Adversarial NetworkImage
🎯 What it does: A storage-efficient visual training framework called SeiT is proposed, which uses 1% pixel storage to compress images into 1024 discrete tokens and directly trains ViT.
Self-Calibrated Cross Attention Network for Few-Shot Segmentation
Qianxiong Xu (Nanyang Technological University), Cheng Long (Nanyang Technological University)
SegmentationTransformerImage
🎯 What it does: This paper proposes a Self-Calibrating Cross-Attention Network (SCCAN) for few-shot semantic segmentation, addressing the issues of mismatched foreground and background matching in support samples and the entanglement of foreground and background.
Self-Evolved Dynamic Expansion Model for Task-Free Continual Learning
Fei Ye (University of York), Adrian G. Bors (University of York)
OptimizationMixture of ExpertsAuto EncoderImage
🎯 What it does: Proposed and implemented a Self-Evolving Dynamic Expansion Model (SEDEM) for task-agnostic continual learning, capable of automatically determining whether to add new experts and conducting incremental training in data streams.
Self-Feedback DETR for Temporal Action Detection
Jihwan Kim (Sungkyunkwan University), Jae-Pil Heo (Sungkyunkwan University)
RecognitionObject DetectionTransformerVideo
🎯 What it does: By adding cross-attention feedback to the self-attention module of DETR, the issue of self-attention collapse in the temporal dimension is addressed;
Self-Ordering Point Clouds
Pengwan Yang (University of Amsterdam), Yuki M. Asano (University of Amsterdam)
ClassificationRetrievalOptimizationComputational EfficiencyContrastive LearningPoint Cloud
🎯 What it does: Developed an unsupervised self-learning method to rank the importance of 3D point clouds, generating a representative subset to enhance the efficiency of subsequent tasks;
Self-Organizing Pathway Expansion for Non-Exemplar Class-Incremental Learning
Kai Zhu (Alibaba Group), Zheng-Jun Zha (University of Science and Technology of China)
ClassificationKnowledge DistillationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: A self-organizing path expansion scheme is proposed, which significantly alleviates the optimization interference between old and new classes by organizing paths in a class-specific manner and guiding feature updates in non-sample class incremental learning.
Self-regulating Prompts: Foundational Model Adaptation without Forgetting
Muhammad Uzair Khattak (Mohamed bin Zayed University of AI), Fahad Shahbaz Khan (Mohamed bin Zayed University of AI)
ClassificationDomain AdaptationPrompt EngineeringContrastive LearningImage
🎯 What it does: Proposed the PromptSRC framework, which utilizes self-regularization to learn prompts on CLIP, balancing task-specific and general features.
Self-similarity Driven Scale-invariant Learning for Weakly Supervised Person Search
Benzhi Wang (Chinese Academy of Sciences), Zhen Lei (Chinese Academy of Sciences)
Object DetectionRetrievalSupervised Fine-TuningImage
🎯 What it does: A Self-similarity driven Scale-invariant Learning (SSL) framework is proposed to address the scale variation problem in weakly supervised person retrieval.
Self-Supervised Burst Super-Resolution
Goutam Bhat, Zhihao Xia
RestorationSuper ResolutionOptical FlowImageVideo
🎯 What it does: A self-supervised training framework is proposed that utilizes only unregistered low-resolution raw burst images to train burst super-resolution networks, thereby eliminating the need for synthetic data or weakly registered high-resolution references.
Self-Supervised Character-to-Character Distillation for Text Recognition
Tongkun Guan (Shanghai Jiao Tong University), Xiaokang Yang (Shanghai Jiao Tong University)
RecognitionSegmentationKnowledge DistillationRepresentation LearningTransformerContrastive LearningImage
🎯 What it does: A self-supervised character-level distillation framework (CCD) is proposed, which treats each character as a basic learning unit for representation learning of text images through self-supervised character segmentation and geometric transformation alignment.
Self-supervised Cross-view Representation Reconstruction for Change Captioning
Yunbin Tu (University of Chinese Academy of Sciences), Qingming Huang (University of Chinese Academy of Sciences)
GenerationRepresentation LearningTransformerContrastive LearningImage
🎯 What it does: This paper proposes a self-supervised cross-view representation reconstruction network called SCORER, which aims to learn robust differential representations and generate differential explanations in the presence of view pseudo-transformations.
Self-supervised Image Denoising with Downsampled Invariance Loss and Conditional Blind-Spot Network
Yeong Il Jang (Seoul National University), Nam Ik Cho (Seoul National University)
RestorationConvolutional Neural NetworkImage
🎯 What it does: A self-supervised image denoising framework based on Conditional Blind Spot Networks (C-BSN) is proposed.
Self-supervised Learning of Implicit Shape Representation with Dense Correspondence for Deformable Objects
Baowen Zhang (Institute of Software, Chinese Academy of Sciences), Hongan Wang (Institute of Software, Chinese Academy of Sciences)
GenerationRepresentation LearningPoint CloudMesh
🎯 What it does: A self-supervised neural implicit shape representation method has been developed, capable of learning dense correspondences for deformable objects and generating editable 3D models.
Self-supervised Learning to Bring Dual Reversed Rolling Shutter Images Alive
Wei Shang (Harbin Institute of Technology), Wangmeng Zuo (Harbin Institute of Technology)
Image TranslationRestorationKnowledge DistillationOptical FlowImageVideo
🎯 What it does: A self-supervised learning-based Dual Reversed Rolling Shutter image correction framework, SelfDRSC, is proposed, which can restore dual reversed rolling shutter images to high frame rate global shutter videos.
Self-Supervised Monocular Depth Estimation by Direction-aware Cumulative Convolution Network
Wencheng Han (University of Macau), Jianbing Shen (Beijing Institute of Technology)
Depth EstimationAutonomous DrivingConvolutional Neural NetworkImage
🎯 What it does: To address the direction sensitivity and environmental dependence in self-supervised monocular depth estimation, a Direction-Aware Cumulative Convolutional Network (DaCCN) is proposed.
Self-supervised Monocular Depth Estimation: Let's Talk About The Weather
Kieran Saunders (Aston University), Luis J. Manso (Aston University)
Depth EstimationAutonomous DrivingTransformerImage
🎯 What it does: A self-supervised monocular depth estimation framework called Robust-Depth is proposed, which maintains robust performance under various adverse weather and lighting conditions.
Self-supervised Monocular Underwater Depth Recovery, Image Restoration, and a Real-sea Video Dataset
Nisha Varghese (Indian Institute of Technology Madras), A. N. Rajagopalan (Indian Institute of Technology Madras)
RestorationDepth EstimationConvolutional Neural NetworkImageVideo
🎯 What it does: A self-supervised monocular underwater depth recovery and image enhancement network named USe-ReDI-Net is proposed, which can simultaneously output depth maps and dehazed enhanced images from a single underwater image.
Self-Supervised Object Detection from Egocentric Videos
Peri Akiva (Rutgers University), Tal Hassner (Meta AI)
Object DetectionTransformerContrastive LearningVideo
🎯 What it does: This paper proposes a self-supervised, category-agnostic object detection framework called DEVI, which learns fine-grained, viewpoint and illumination invariant features directly from unlabeled first-person videos and generates bounding boxes on single frames.
Self-supervised Pre-training for Mirror Detection
Jiaying Lin (City University of Hong Kong), Rynson W.H. Lau (City University of Hong Kong)
ClassificationRepresentation LearningConvolutional Neural NetworkSupervised Fine-TuningContrastive LearningImage
🎯 What it does: A three-stage self-supervised pre-training framework (image-level, block-level, pixel-level) is designed specifically to extract mirror reflection features for the mirror detection task and fine-tuned on mirror detection baselines (MirrorNet, VCNet).
Semantic Attention Flow Fields for Monocular Dynamic Scene Decomposition
Yiqing Liang (Brown University), James Tompkin (Brown University)
Object DetectionSegmentationTransformerNeural Radiance FieldOptical FlowVideo
🎯 What it does: A 4D neural volume representation based on monocular video (SAFF) has been constructed, capable of simultaneously reconstructing time-varying color, density, scene flow, semantic, and attention information, and utilizing this information for unsupervised segmentation and editing of dynamic scenes.
Semantic Information in Contrastive Learning
Shengjiang Quan (University of Tokyo), Yuji Yamakawa (University of Tokyo)
Object DetectionSegmentationDepth EstimationConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: In the pre-training phase, the SemCL method is proposed, which generates contrastive samples of objects and their surrounding environments by utilizing publicly available semantic annotations to construct a new contrastive learning pretext task;
Semantic-Aware Dynamic Parameter for Video Inpainting Transformer
Eunhye Lee (Hanyang University), Tae Hyun Kim (Hanyang University)
RestorationTransformerMixture of ExpertsVideo
🎯 What it does: A video inpainting framework based on Transformer, SAVIT, is proposed, which can utilize semantic information in video frames to achieve more accurate local reconstruction.
Semantic-Aware Implicit Template Learning via Part Deformation Consistency
Sihyeon Kim (Korea University), Hyunwoo J. Kim (Korea University)
SegmentationGenerationAuto EncoderPoint Cloud
🎯 What it does: Learning a semantic-aware implicit template network that achieves unified representation and high-quality correspondence for different shapes through self-supervised segmentation feature-guided templates and deformation fields;
Semantically Structured Image Compression via Irregular Group-Based Decoupling
Ruoyu Feng (University of Science and Technology of China), Zhibo Chen (University of Science and Technology of China)
Object DetectionSegmentationPose EstimationCompressionTransformerImage
🎯 What it does: This paper proposes a semantic structure image compression method based on irregular group segmentation, which divides the image into several irregular regions using a custom group mask and compresses each group independently, thereby supporting selective transmission and reconstruction of only the regions of interest when needed.
Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos
Rui Qian (Chinese University of Hong Kong), Dahua Lin (Chinese University of Hong Kong)
Object DetectionSegmentationTransformerVideo
🎯 What it does: A self-supervised framework SMTC is proposed, which combines high-level semantics and low-level temporal correspondence to achieve unsupervised video object learning, utilizing semantic-aware mask slot attention for semantic decomposition and instance recognition.
Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning
Kaiyou Song (Megvii Technology), Jin Xie (Megvii Technology)
Object DetectionSegmentationRepresentation LearningTransformerContrastive LearningImage
🎯 What it does: This paper proposes a method that utilizes Semantic Consistent Feature Search (SCFS) in self-supervised contrastive learning to adaptively find semantically consistent feature regions for comparison, thereby alleviating the semantic inconsistency issues caused by data augmentation and enhancing the model's focus on target regions.
Semantify: Simplifying the Control of 3D Morphable Models Using CLIP
Omer Gralnik, Ariel Shamir
GenerationData SynthesisVision Language ModelContrastive LearningMesh
🎯 What it does: By using a self-supervised approach, the semantic information of CLIP is mapped to the parameters of the 3D Morphable Model (3DMM), enabling direct control of the 3D model using semantic descriptors.
SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Driving
Shuai Yuan (Duke University), Carlo Tomasi (Duke University)
SegmentationAutonomous DrivingOptimizationOptical FlowImageVideo
🎯 What it does: In unsupervised optical flow estimation, semantic segmentation information is incorporated to improve the network structure and enhance optical flow accuracy.
Semi-Supervised Learning via Weight-Aware Distillation under Class Distribution Mismatch
Pan Du (Renmin University of China), Hong Chen (Renmin University of China)
Knowledge DistillationRepresentation LearningContrastive LearningImage
🎯 What it does: This paper proposes a semi-supervised learning framework based on Weight-Aware Distillation (WAD) to address scenarios with class distribution mismatch.
Semi-Supervised Semantic Segmentation under Label Noise via Diverse Learning Groups
Peixia Li (University of Sydney), Anton van den Hengel (Amazon)
SegmentationConvolutional Neural NetworkTransformerImageBiomedical DataComputed Tomography
🎯 What it does: The study proposes a robust training framework for semi-supervised semantic segmentation tasks in the presence of pixel-level label noise.
Semi-supervised Semantics-guided Adversarial Training for Robust Trajectory Prediction
Ruochen Jiao (Northwestern University), Qi Zhu (Northwestern University)
Autonomous DrivingAdversarial AttackGraph Neural NetworkAuto EncoderGenerative Adversarial NetworkTime Series
🎯 What it does: A semi-supervised semantic-guided adversarial training method has been developed to enhance the robustness and generalization ability of trajectory prediction models under adversarial attacks.
Semi-supervised Speech-driven 3D Facial Animation via Cross-modal Encoding
Peiji Yang (Huawei), Zhisheng Wang (Tencent)
GenerationData SynthesisTransformerGenerative Adversarial NetworkVideoMeshAudio
🎯 What it does: This paper proposes a cross-modal semi-supervised framework for generating speech-driven 3D facial animations.
SEMPART: Self-supervised Multi-resolution Partitioning of Image Semantics
Sriram Ravindran (Adobe), Debraj Basu (Adobe)
Object DetectionSegmentationTransformerContrastive LearningImage
🎯 What it does: This paper proposes a self-supervised multi-resolution image semantic segmentation method called SEMPART based on DINO features, which can simultaneously generate coarse and fine foreground masks.
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning
Haoyu He (Monash University), Bohan Zhuang (Monash University)
ClassificationSegmentationTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes a Sensitivity-aware visual Parameter-Efficient Fine-Tuning (SPT) method that can adaptively identify and allocate task-specific important parameter locations within a given parameter budget, significantly improving PEFT performance.