ECCV 2024 Papers — Page 20
European Conference on Computer Vision · 2387 papers
Self-Cooperation Knowledge Distillation for Novel Class Discovery
Yuzheng Wang (Fudan University), Lizhe Qi (Fudan University)
ClassificationKnowledge DistillationConvolutional Neural NetworkTransformerImage
🎯 What it does: Proposed a self-coordinated knowledge distillation (SCKD) method to address the imbalance between known and unknown class samples in novel class discovery (NCD), leveraging each training sample to simultaneously review known classes and discover unknown classes.
Self-Guided Generation of Minority Samples Using Diffusion Models
Soobin Um (KAIST), Jong Chul Ye (KAIST)
Data SynthesisDiffusion modelImage
🎯 What it does: Propose a self-supervised minority class sample generation method that uses only pre-trained diffusion models, leveraging self-guided sampling to induce generation of samples in low-density regions.
Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance
Donghoon Ahn (Korea University), Seungryong Kim (Korea University)
RestorationGenerationDiffusion modelImage
🎯 What it does: This paper proposes a novel sampling guidance method called Perturbed-Attention Guidance (PAG), which generates intermediate samples with structural degradation by replacing the self-attention matrix in the diffusion U-Net with an identity matrix, and guides the model to avoid these degraded samples during the reverse sampling process, thereby improving the quality of both unconditional and conditional image generation.
Self-Supervised Any-Point Tracking by Contrastive Random Walks
Ayush Shrivastava (University of Michigan), Andrew Owens (University of Michigan)
Object TrackingTransformerContrastive LearningVideo
🎯 What it does: Propose a fully self-supervised arbitrary point tracking method that utilizes a global matching Transformer to achieve temporal consistency tracking through contrastive random walks.
Self-Supervised Audio-Visual Soundscape Stylization
Tingle Li (University of California, Berkeley), Gopala Krishna Anumanchipalli (University of Michigan)
GenerationData SynthesisVision Language ModelDiffusion modelAuto EncoderVideoMultimodalityAudio
🎯 What it does: This paper proposes a self-supervised audio-visual soundscape stylization method, which can adjust the timbre of input speech and environmental noise to match a target scene's soundscape by training on unlabeled internet videos.
Self-supervised co-salient object detection via feature correspondences at multiple scales
Souradeep Chakraborty (Stony Brook University), Dimitris Samaras (Stony Brook University)
SegmentationTransformerContrastive LearningImageBenchmark
🎯 What it does: Propose a two-stage self-supervised co-salient object detection method named SCoSPARC, leveraging the correspondence between local and global scale features of ViT to segment co-occurring salient objects in image groups.
Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection
Yuanpeng Tu (Tongji University), cairong zhao
Domain AdaptationAnomaly DetectionTransformerContrastive LearningImagePoint Cloud
🎯 What it does: Proposes a self-supervised local-to-global feature adaptation framework (LSFA) to adapt features from pre-trained models to 3D industrial defect detection tasks, improving detection performance through cross-modal alignment and single-modal compression.
Self-Supervised Representation Learning for Adversarial Attack Detection
Yi Li (Lancaster University), Neeraj Suri (Lancaster University)
Representation LearningAdversarial AttackConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: Proposed a self-supervised representation learning framework for adversarial attack detection, trained using three losses: pixel mapping, prototype contrast estimation, and instance contrast learning, and designed a parallel axial attention (PAA) encoder.
Self-supervised Shape Completion via Involution and Implicit Correspondences
Mengya Liu (ETH Zurich), Federico Tombari (Google)
RestorationGenerationPoint Cloud
🎯 What it does: Propose a self-supervised 3D shape completion framework that leverages shape correspondence and self-inverse function constraints to recover complete geometry from incomplete scans.
Self-Supervised Underwater Caustics Removal and Descattering via Deep Monocular SLAM
Jonathan Sauder (EPFL), Devis Tuia (EPFL)
RestorationSegmentationDepth EstimationSimultaneous Localization and MappingImageVideo
🎯 What it does: Proposed CausticsNet and BackscatterNet, single-frame underwater illumination (caustics and backscatter) removal networks based on self-supervised monocular SLAM, capable of real-time underwater image restoration without requiring ground truth labels.
Self-Supervised Video Copy Localization with Regional Token Representation
Minlong Lu (Ant Group), Xiaobo Zhang (Ant Group)
Object DetectionRetrievalRepresentation LearningTransformerContrastive LearningVideo
🎯 What it does: Proposed a self-supervised video duplication localization framework that extracts local features by introducing Regional Tokens in Vision Transformers and trains a temporal localization model using self-supervised generated video pairs
Self-Supervised Video Desmoking for Laparoscopic Surgery
Renlong Wu (Harbin Institute of Technology), Wangmeng Zuo (Harbin Institute of Technology)
RestorationConvolutional Neural NetworkTransformerOptical FlowVideoBiomedical Data
🎯 What it does: Proposes a self-supervised surgical video defogging (SelfSVD) method that utilizes previous fog-free frames in the video as unaligned supervision and reference, enabling training and online inference without paired data.
Self-supervised visual learning from interactions with objects
Arthur Aubret (Frankfurt Institute for Advanced Studies), Jochen Triesch (Frankfurt Institute for Advanced Studies)
ClassificationRecognitionRepresentation LearningVision-Language-Action ModelContrastive LearningVideo
🎯 What it does: This paper proposes an action-aware self-supervised learning method (AA-SSL) that leverages action information generated through object interaction. By aligning action embeddings with corresponding image embeddings, the method enhances the category generalization capability of visual representations.
Self-Training Room Layout via Geometry-aware Ray-casting
Bolivar Solarte (National Tsing Hua University), Min Sun (National Tsing Hua University)
SegmentationConvolutional Neural NetworkPoint CloudMesh
🎯 What it does: Proposes a geometry-aware self-training framework that utilizes ray projection to aggregate multi-view estimates, generating pseudo labels to enhance room layout estimation in unlabeled scenes.
SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes
Mohammad Zohaib (Italian Institute of Technology), Alessio Del Bue (Italian Institute of Technology)
Pose EstimationConvolutional Neural NetworkPoint Cloud
🎯 What it does: Propose the SelfGeo method, which utilizes self-supervised learning to estimate repeatable, semantically consistent 3D keypoints in point cloud sequences, ensuring the relative positions of keypoints remain unchanged during non-rigid deformations through geodesic distance constraints.
SelfSwapper: Self-Supervised Face Swapping via Shape Agnostic Masked AutoEncoder
Jaeseong Lee (Korea Advanced Institute of Science and Technology), Jaegul Choo (Korea Advanced Institute of Science and Technology)
Image TranslationGenerationConvolutional Neural NetworkAuto EncoderImageMesh
🎯 What it does: Propose SelfSwapper, achieving self-supervised face swapping through Shape Agnostic Masked AutoEncoder (SAMAE), which can precisely preserve source identity features while maintaining non-identity attributes of the target face.
Semantic Diversity-aware Prototype-based Learning for Unbiased Scene Graph Generation
Jaehyeong Jeon (Korea Advanced Institute of Science and Technology), Chanyoung Park (Korea Advanced Institute of Science and Technology)
GenerationMultimodality
🎯 What it does: This paper proposes a model-agnostic semantic diversity-aware prototype learning framework (DPL) for scene graph generation tasks, which captures multiple semantics of the same predicate by learning prototypes and their distributions in the semantic space to achieve unbiased predictions.
Semantic Residual Prompts for Continual Learning
Martin Menabue (University of Modena and Reggio Emilia), SIMONE CALDERARA (University of Modena and Reggio Emilia)
ClassificationTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: Propose a two-layer semantic residual prompt continual learning method called STAR-Prompt, which utilizes a frozen CLIP text encoder to generate class prototypes and employs them as keys to retrieve the second-layer prompt, injecting semantic residuals into the frozen ViT, balancing model stability and plasticity.
Semantic-guided Robustness Tuning for Few-Shot Transfer Across Extreme Domain Shift
kangyu xiao, junjie li
ClassificationDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningImageMultimodalityBenchmark
🎯 What it does: Propose a Semantic-Guided Robustness Tuning (SRT) framework for fine-tuning large pre-trained models (LPM) in cross-domain few-shot classification (CDFSC) tasks, comprising two modules: modulus-matched image-text hybrid (MMIT-Mixup) and robustness-invariance fine-tuning (RI-FT).
Semantically Guided Representation Learning For Action Anticipation
Anxhelo Diko (Sapienza University of Rome), Luigi Cinque (Sapienza University of Rome)
RecognitionRepresentation LearningTransformerLarge Language ModelVision-Language-Action ModelContrastive LearningVideoMultimodality
🎯 What it does: This paper proposes a framework named S-GEAR for action prediction by learning visual action prototypes and combining them with a language model.
SemanticHuman-HD: High Resolution Semantic disentangled 3D Human Generation
Peng Zheng (Jilin University), Rui Ma (Jilin University)
GenerationSuper ResolutionNeural Radiance FieldGenerative Adversarial NetworkImage
🎯 What it does: Proposed SemanticHuman-HD, achieving high-resolution (1024×1024) and semantically disentangled 3D human image generation.
SemGrasp: Semantic Grasp Generation via Language Aligned Discretization
Kailin Li (Shanghai Jiao Tong University), Bo Dai (Shanghai AI Laboratory)
Robotic IntelligenceTransformerLarge Language ModelSupervised Fine-TuningVision-Language-Action ModelAuto EncoderTextMultimodalityPoint Cloud
🎯 What it does: Propose the SemGrasp method, generating natural human grasping poses using semantically aligned discretized representations.
Semi-supervised Segmentation of Histopathology Images with Noise-Aware Topological Consistency
Meilong Xu (Stony Brook University), Chao Chen (Stony Brook University)
SegmentationImageBiomedical Data
🎯 What it does: This paper proposes a semi-supervised segmentation framework (TopoSemiSeg), which enables the model to learn correct topological structures on unlabeled images through a noise-aware topological consistency loss, thereby improving accuracy in dense distribution gland/cell nucleus segmentation tasks.
Semi-Supervised Teacher-Reference-Student Architecture for Action Quality Assessment
Wulian Yun (Beijing University of Posts and Telecommunications), Huadong Ma (Beijing University of Posts and Telecommunications)
RecognitionKnowledge DistillationConvolutional Neural NetworkTransformerVideo
🎯 What it does: Proposes a semi-supervised teacher-reference-student framework to learn in action quality assessment (AQA) tasks using only a small amount of labeled data and leveraging a large amount of unlabeled data.
Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization
Hongtao Wu (Hong Kong University of Science and Technology), Lei Zhu (Hong Kong University of Science and Technology)
RestorationConvolutional Neural NetworkTransformerMixture of ExpertsContrastive LearningVideo
🎯 What it does: Propose a semi-supervised video de-snowing network, SemiVDN, which leverages unlabeled real snow videos to enhance generalization and introduces temporal expert modules and distribution-driven contrastive regularization;
Semicalibrated Relative Pose from an Affine Correspondence and Monodepth
Petr Hruby (ETH Zürich), Daniel Barath (ETH Zürich)
Pose EstimationDepth EstimationImage
🎯 What it does: This paper proposes a method for semi-calibrated relative pose estimation using a single affine correspondence and monocular depth prediction.
SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance
Lukas Hoyer (ETH Zurich), Federico Tombari (Google)
SegmentationTransformerSupervised Fine-TuningVision Language ModelImageText
🎯 What it does: This paper proposes a semi-supervised semantic segmentation framework called SemiVL based on vision-language models (VLM), aiming to achieve high-quality pixel-level segmentation with only a very limited number of labeled images by leveraging the semantic prior of VLM.
SemReg: Semantics Constrained Point Cloud Registration
Sheldon Fung (La Trobe University), HONGDONG LI
Pose EstimationConvolutional Neural NetworkTransformerSupervised Fine-TuningContrastive LearningGaussian SplattingMultimodalityPoint Cloud
🎯 What it does: Propose the SemReg framework, which leverages semantic information from 2D images to generate more robust 3D features for point cloud registration.
SemTrack: A Large-scale Dataset for Semantic Tracking in the Wild
Pengfei Wang (Singapore University Of Technology And Design), Jun Liu (Singapore University Of Technology And Design)
RecognitionObject DetectionObject TrackingMeta LearningConvolutional Neural NetworkRecurrent Neural NetworkTransformerVision Language ModelVideoTextBenchmark
🎯 What it does: Proposes SemTrack, a novel semantic tracking dataset and the corresponding SemTracker model, capable of locating, tracking, and recording the interaction trajectories of targets in videos;
SENC: Handling Self-collision in Neural Cloth Simulation
Zhouyingcheng Liao (University of Hong Kong), Taku Komura (University of Hong Kong)
Graph Neural NetworkMeshSequentialPhysics Related
🎯 What it does: Propose SENC, a self-supervised neural fabric simulation method that obtains penetration volume as self-collision loss through Global Cross-Analysis (GIA), and constructs a self-collision-aware graph neural network (with added spatial distance edges) along with an adjustable external force scheme to efficiently handle self-collisions in fabrics.
Sequential Representation Learning via Static-Dynamic Conditional Disentanglement
Mathieu Cyrille Simon (Uclouvain), Christophe De Vleeschouwer (Uclouvain)
Representation LearningFlow-based ModelAuto EncoderVideo
🎯 What it does: This paper proposes a novel static-dynamic conditional decoupling representation learning framework that can separate time-invariant factors and time-varying factors in video sequences under unsupervised conditions;
SFPNet: Sparse Focal Point Network for Semantic Segmentation on General LiDAR Point Clouds
Yanbo Wang (Shanghai Jiao Tong University), Weidong Chen (Shanghai Jiao Tong University)
SegmentationAutonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: Proposes the Sparse Focal Point Network (SFPNet), which replaces window attention with a sparse focal point modulation (SFPM) module for semantic segmentation of various types of LiDAR point clouds.
SG-NeRF: Neural Surface Reconstruction with Scene Graph Optimization
Yiyang Chen (Zhejiang University), Yanchao Yang (University of Hong Kong)
Pose EstimationOptimizationNeural Radiance FieldImageMesh
🎯 What it does: Propose a joint optimization framework called SG-NeRF based on scene graphs, which trains neural radiance fields (NeRF) while simultaneously optimizing camera poses and confidence levels, achieving high-quality 3D surface reconstruction even in the presence of significant camera pose noise.
SGS-SLAM: Semantic Gaussian Splatting For Neural Dense SLAM
Mingrui Li (Dalian University of Technology), Hongyu Wang (Shanghai Jiao Tong University)
SegmentationPose EstimationDepth EstimationGaussian SplattingSimultaneous Localization and MappingImagePoint Cloud
🎯 What it does: This paper proposes a semantic visual SLAM system called SGS-SLAM based on 3D Gaussian splatting, achieving real-time dense reconstruction and semantic segmentation.
Shape from Heat Conduction
Sriram Narayanan (Carnegie Mellon University), Srinivasa G. Narasimhan (Carnegie Mellon University)
Depth EstimationVideoPhysics Related
🎯 What it does: Propose a method to recover 3D object shapes from thermal videos using thermal imaging and the heat conduction equation.
Shape-guided Configuration-aware Learning for Endoscopic-image-based Pose Estimation of Flexible Robotic Instruments
Yiyao Ma (Chinese University of Hong Kong), Qi Dou (Chinese University of Hong Kong)
Pose EstimationTransformerImagePoint CloudBiomedical Data
🎯 What it does: Propose a shape-guided configuration-aware learning framework that enhances feature extraction for the pose of flexible robots in monocular endoscopic images by leveraging 3D shape priors, thereby achieving pose regression;
Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Data
Tuo Feng (University of Technology Sydney), Yi Yang (Zhejiang University)
ClassificationObject DetectionSegmentationData SynthesisRepresentation LearningConvolutional Neural NetworkContrastive LearningPoint Cloud
🎯 What it does: This paper proposes the Shape2Scene method, which pretrains 3D scene representations using shape data and performs unsupervised learning on multi-scale high-resolution networks (MH-P, MH-V).
Shapefusion: 3D localized human diffusion models
Rolandos Alexandros Potamias (Imperial College London), Stefanos Zafeiriou (Imperial College London)
GenerationGraph Neural NetworkDiffusion modelMesh
🎯 What it does: Proposed a local shape editing framework based on 3D diffusion models, which can achieve precise and controllable modifications to any selected region while preserving the integrity of unedited areas.
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction
Zekun Qi (Xi'an Jiaotong University), Kaisheng Ma (IIIS, Tsinghua University)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningVision-Language-Action ModelTextMultimodalityPoint CloudMeshBenchmark
🎯 What it does: Proposed ShapeLLM, a multi-modal LLM based on the ReCon++ 3D encoder for achieving global 3D object understanding and embodied interaction, and created the 3D MM-Vet benchmark.
ShareGPT4V: Improving Large Multi-Modal Models with Better Captions
Lin Chen (University of Science and Technology of China), Dahua Lin (Chinese University of Hong Kong)
TransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: This paper constructs a dataset with detailed image captions (ShareGPT4V and ShareGPT4V-PT), and trains ShareGPT4V-7B on this dataset to study the impact of training data on modal alignment in large multimodal models (LMM).
Shedding More Light on Robust Classifiers under the lens of Energy-based Models
Mujtaba Hussain Mirza (Sapienza University of Rome), Iacopo Masi (Sapienza University of Rome)
ClassificationAdversarial AttackImageBenchmark
🎯 What it does: This paper reinterprets robust classifiers as energy models, deeply analyzes the dynamics of the energy landscape during adversarial training, and proposes a weighted energy adversarial training (WEAT) method based on this analysis.
SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning
Haiwen Diao (Dalian University of Technology), Long Chen (HKUST)
Computational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningImageVideoTextMultimodality
🎯 What it does: Proposes a novel resource-constrained transfer learning framework called SHERL, which employs a two-phase separation strategy of early aggregation and late regulation, significantly reducing GPU memory demand during fine-tuning while maintaining or improving performance.
SHIC: Shape-Image Correspondences with no Keypoint Supervision
Aleksandar Shtedritski (University of Oxford), Andrea Vedaldi (University of Oxford)
GenerationPose EstimationTransformerDiffusion modelContrastive LearningImageMesh
🎯 What it does: This study proposes a fully unsupervised shape-image correspondence learning method called SHIC, which can automatically generate high-quality image-template correspondences under the condition of only being given a 3D template and a small number of masked images.
Shifted Autoencoders for Point Annotation Restoration in Object Counting
Yuda Zou (Wuhan University), Yongchao Xu (Wuhan University)
RestorationConvolutional Neural NetworkAuto EncoderPoint Cloud
🎯 What it does: Propose Shifted Autoencoders (SAE), which directly improves the consistency of point annotations by randomly translating point annotations and restoring them to their original positions before training the counting model.
SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Grounding
Zixu Cheng (Queen Mary University of London), Yu Kong (Michigan State University)
RetrievalTransformerLarge Language ModelVision Language ModelVideo
🎯 What it does: Proposes the SHINE method, which generates semantically feasible hard negative samples using large language models and enhances the combinatorial generalization ability of video temporal retrieval through a coarse-to-fine two-level significance ranking.
ShoeModel: Learning to Wear on the User-specified Shoes via Diffusion Model
Wenyu Li (Harbin Institute of Technology), Wangmeng Zuo (Harbin Institute of Technology)
SegmentationGenerationPose EstimationDiffusion modelImage
🎯 What it does: Designed and implemented a system called ShoeModel to generate ultra-realistic advertising images of user-specified shoe models interacting with human legs while maintaining shoe identity consistency.
Siamese Vision Transformers are Scalable Audio-visual Learners
Yan-Bo Lin (University of North Carolina at Chapel Hill), Gedas Bertasius (University of North Carolina at Chapel Hill)
ClassificationRetrievalRepresentation LearningTransformerAuto EncoderContrastive LearningVideoMultimodalityAudio
🎯 What it does: Proposes an audio-visual twin network (AVSiam) that uses a shared vision Transformer to uniformly process audio and visual inputs, and performs self-supervised pre-training through multi-scale random masking, contrastive matching, and reconstruction objectives.
SIGMA: Sinkhorn-Guided Masked Video Modeling
Mohammadreza Salehi (University of Amsterdam), Yuki M Asano
Representation LearningTransformerImageVideo
🎯 What it does: Proposed the SIGMA framework, which enhances video representation learning by generating semantic deep feature targets through introducing Sinkhorn-regularized clustering in video mask modeling.
SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark
Zhengdi Yu (Imperial College London), Tolga Birdal (Imperial College London)
GenerationData SynthesisTransformerVision Language ModelAuto EncoderVideoMultimodalityMeshBenchmark
🎯 What it does: Constructed and released SignAvatars—the first large-scale multi-prompt 3D sign language full-body motion dataset, providing complete 3D avatar mesh annotations.
SignGen: End-to-End Sign Language Video Generation with Latent Diffusion
Fan Qi (Tianjin University of Technology), Huaiwen Zhang (Tianjin University of Technology)
GenerationPose EstimationDepth EstimationVision Language ModelDiffusion modelOptical FlowVideoTextMultimodality
🎯 What it does: Developed an end-to-end text-to-sign language video generation system called SignGen based on Latent Diffusion Models (LDMs), directly mapping textual descriptions to complete sign language videos (including body, hand, and facial expressions), eliminating traditional intermediate steps such as gloss or pose prediction.
SILC: Improving Vision Language Pretraining with Self-Distillation
Muhammad Ferjad Naeem (ETH Zurich), Federico Tombari (Google)
ClassificationObject DetectionSegmentationRetrievalKnowledge DistillationRepresentation LearningTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: Proposed the SILC framework, combining image-text contrastive learning with self-distillation to enhance the performance of vision-language models on global and local features.
Similarity of Neural Architectures using Adversarial Attack Transferability
Jaehui Hwang (Yonsei University), Jong-Seok Lee
Knowledge DistillationRepresentation LearningAdversarial AttackImage
🎯 What it does: This paper proposes a similarity metric called SAT based on the transfer rate of adversarial attacks, conducts a large-scale quantitative analysis on 69 ImageNet classifiers, and explores its impact on model ensembling and distillation.
SimPB: A Single Model for 2D and 3D Object Detection from Multiple Cameras
Yingqi Tang (Nullmax), Erkang Cheng (Nullmax)
Object DetectionAutonomous DrivingTransformerVideo
🎯 What it does: Propose a unified query-based framework called SimPB, which can simultaneously perform Perspective 2D object detection and Bird's-eye view (BEV) 3D object detection under multi-camera inputs;
Simple Unsupervised Knowledge Distillation With Space Similarity
Aditya Singh, Haohan Wang (University of Illinois Urbana Champaign)
ClassificationObject DetectionSegmentationRetrievalKnowledge DistillationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: What they did: Propose a new unsupervised knowledge distillation method that directly aligns the teacher and student's embedding manifolds through spatial similarity loss.
Simplifying Source-Free Domain Adaptation for Object Detection: Effective Self-Training Strategies and Performance Insights
Yan Hao (EPFL), Olga Fink (EPFL)
Object DetectionDomain AdaptationConvolutional Neural NetworkImage
🎯 What it does: For target detection domain adaptation when source data is unavailable, a simpler self-training strategy is proposed and its effectiveness is evaluated against existing complex methods.
SINDER: Repairing the Singular Defects of DINOv2
Haoqi Wang (EPFL), Mathieu Salzmann (EPFL)
ClassificationRestorationSegmentationTransformerSupervised Fine-TuningImage
🎯 What it does: Investigated and repaired the single-value defect in the DINOv2 Vision Transformer, proposing the SINDER method which fixes the defect by fine-tuning singular values and incorporating smooth regularization.
Single-Mask Inpainting for Voxel-based Neural Radiance Fields
Jiafu Chen (Zhejiang University), Lei Zhao (Zhejiang University)
RestorationGenerationDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderImage
🎯 What it does: Proposes a method to remove 3D scene objects and fill the background using only a single-view mask, projecting the mask into the NeRF voxel space, then generating color and depth priors through a 2D inpainter, followed by refinement using a 2D Diffusion model across all views.
Single-Photon 3D Imaging with Equi-Depth Photon Histograms
Kaustubh Sadekar (Portland State University), Atul Ingle (Portland State University)
Depth EstimationImagePhysics Related
🎯 What it does: This paper proposes a single-photon 3D imaging method based on equal-depth histograms (ED histograms), utilizing a proportional-step equal-depth histogram (PEDH) to estimate arbitrary quantiles online, thereby achieving a low-bandwidth and low-pixel-memory imaging process.
SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers
Nanye Ma (New York University), Saining Xie (New York University)
GenerationTransformerDiffusion modelFlow-based ModelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Building upon Diffusion Transformers, we propose Scalable Interpolant Transformers (SiT), incorporating improvements such as interpolators, continuous time training, velocity prediction, and adjustable noise coefficients to create more flexible flow/diffusion generation models, achieving superior FID results on ImageNet 256×256 and 512×512.
Situated Instruction Following
So Yeon Min (Carnegie Mellon University), Roozbeh Mottaghi (Fair, Meta)
Robotic IntelligenceTransformerLarge Language ModelPrompt EngineeringVision-Language-Action ModelMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Introduce a novel situated instruction following benchmark that emphasizes ambiguity, temporality, and dynamics within tasks.
Six-Point Method for Multi-Camera Systems with Reduced Solution Space
Banglei Guan (National University of Defense Technology), Laurent Kneip (ShanghaiTech University)
Pose EstimationImage
🎯 What it does: Proposed a minimal solver for solving relative pose in multi-camera systems using six point correspondences (PC), enabling 6DOF pose estimation;
SkateFormer: Skeletal-Temporal Transformer for Human Action Recognition
Jeonghyeok Do (Korea Advanced Institute of Science and Technology), Munchurl Kim (Korea Advanced Institute of Science and Technology)
RecognitionConvolutional Neural NetworkGraph Neural NetworkTransformerGraphTime Series
🎯 What it does: Proposes a Transformer framework called SkateFormer based on skeleton-temporal partitioned self-attention for human action recognition using skeleton sequences.
Skeleton Recall Loss for Connectivity Conserving and Resource Efficient Segmentation of Thin Tubular Structures
Yannick Kirchhoff (German Cancer Research Center), Klaus H. Maier-Hein
SegmentationComputational EfficiencyConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: Propose Skeleton Recall Loss, a loss function that enforces connectivity constraints on thin tubular structures through a precomputed tubed skeleton.
Skeleton-based Group Activity Recognition via Spatial-Temporal Panoramic Graph
Zhengcen Li (Harbin Institute of Technology), Jingyong Su (Harbin Institute of Technology)
RecognitionObject TrackingPose EstimationGraph Neural NetworkSupervised Fine-TuningVideoGraph
🎯 What it does: Propose a panoramic graph structure combining multi-person human skeletons and object key points, and design a multi-scale spatiotemporal GCN (MP-GCN) for collective action recognition.
Sketch2Vox: Learning 3D Reconstruction from a Single Monocular Sketch Image
Fei Wang (Shantou University)
Image TranslationConvolutional Neural NetworkAuto EncoderImagePoint CloudMesh
🎯 What it does: This study proposes the Sketch2Vox network to reconstruct voxel 3D models from single-view hand-drawn sketches.
Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation
Yingshan Chang (Carnegie Mellon University), Feng Gao (Amazon)
GenerationConvolutional Neural NetworkTransformerDiffusion modelImageTextMultimodality
🎯 What it does: This paper defines and quantifies the completeness and balance of the 'role-filler' relationship in text-image generation, systematically investigates the impact of data distribution skew on model generalization using synthetic and real images (What'sUp benchmark), and proposes two types of statistical metrics for visual and language spaces.
SkyMask: Attack-agnostic Robust Federated Learning with Fine-grained Learnable Masks
Peishen Yan (Shanghai Jiao Tong University), Haibing Guan (Shanghai Jiao Tong University)
Federated LearningConvolutional Neural NetworkImage
🎯 What it does: Propose SkyMask, a federated learning framework that detects and defends against Byzantine attacks at the server side by utilizing learnable fine-grained masks.
SkyScenes: A Synthetic Dataset for Aerial Scene Understanding
Sahil S Khose, Prithvijit Chattopadhyay (Georgia Institute of Technology)
SegmentationData SynthesisDepth EstimationConvolutional Neural NetworkImageMultimodalityBenchmark
🎯 What it does: Generate an aerial image dataset named SkyScenes containing 33,600 images captured from multiple perspectives, weather conditions, heights, and angles using the Carla simulator, along with 28 classes of semantic, instance, and depth dense annotations.
SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Traffic
Kashyap Chitta (University of Tübingen), Andreas Geiger (University of Tübingen)
Autonomous DrivingConvolutional Neural NetworkTransformerDiffusion modelAuto Encoder
🎯 What it does: This paper proposes SLEDGE, a generative model-based and rule-driven traffic driving environment synthesis and simulation framework;
SLIM: Spuriousness Mitigation with Minimal Human Annotations
Xiwei Xuan (University of California, Davis), Kwan-Liu Ma (University of California, Davis)
ClassificationDomain AdaptationConvolutional Neural NetworkImage
🎯 What it does: Propose the SLIM method, which enhances model robustness against spuriousness by constructing an attention space, using minimal manual attention annotations, and selecting feature-balanced data.
SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flow
Yuanzhi Zhu (ETH Zurich), Qiang Liu (UT Austin)
GenerationKnowledge DistillationFlow-based ModelRectified FlowImage
🎯 What it does: Develop the SlimFlow framework to train smaller, faster inference one-step diffusion models
SlotLifter: Slot-guided Feature Lifting for Learning Object-Centric Radiance Fields
Yu Liu (State Key Laboratory of General Artificial Intelligence), Siyuan Huang (State Key Laboratory of General Artificial Intelligence)
SegmentationGenerationTransformerNeural Radiance FieldImage
🎯 What it does: Propose SlotLifter, a slot-guided feature enhancement method that enables unsupervised learning of 3D object-centric radiance fields, achieving simultaneous scene reconstruction, decomposition, and novel view synthesis.
SmartControl: Enhancing ControlNet for Handling Rough Visual Conditions
Xiaoyu Liu (Harbin Institute of Technology), Wangmeng Zuo (Institute for Intelligent Computing)
GenerationVision Language ModelDiffusion modelImageText
🎯 What it does: Proposes the SmartControl method, which introduces an adaptive control scale predictor on top of text generation control, enabling compatibility between text prompts and visual conditions under rough visual conditions.
SMFANet: A Lightweight Self-Modulation Feature Aggregation Network for Efficient Image Super-Resolution
mingjun zheng (Nanjing University of Science and Technology), Jinshan Pan (Nanjing University of Science and Technology)
Super ResolutionConvolutional Neural NetworkImage
🎯 What it does: Propose a lightweight self-modulated feature aggregation network, SMFANet, for efficient image super-resolution.
SMILe: Leveraging Submodular Mutual Information For Robust Few-Shot Object Detection
Anay Majee (University of Texas at Dallas), Rishabh Iyer (University of Texas at Dallas)
Object DetectionImage
🎯 What it does: Propose a loss framework called SMILe based on submodular mutual information and total submodular information, specifically designed to address class confusion and catastrophic forgetting issues in few-shot object detection.
SMooDi: Stylized Motion Diffusion Model
Lei Zhong (Stability AI), Huaizu Jiang (Northeastern University)
GenerationData SynthesisDiffusion modelTextSequential
🎯 What it does: Propose SMooDi, a stylized motion generation framework that leverages pre-trained text-to-motion diffusion models, capable of rapidly synthesizing diverse and realistic stylized motions based on content text and style motion sequences.
SNeRV: Spectra-preserving Neural Representation for Video
Jina Kim (Ewha Womans University), Jewon Kang
RestorationCompressionRepresentation LearningNeural Radiance FieldVideo
🎯 What it does: Proposed SNeRV, which utilizes discrete wavelet transform (DWT) to separate low-frequency and high-frequency components, encodes only the low-frequency components, and reconstructs high-frequency details through MFU/HFR. Further, it extends the time-domain DWT to capture motion, achieving high-quality reconstruction of video implicit representations.
SNP: Structured Neuron-level Pruning to Preserve Attention Scores
KyungHwan Shim, Shinkook Choi (Nota Inc.)
ClassificationComputational EfficiencyTransformerImage
🎯 What it does: Propose a graph-aware neuron-level pruning method called SNP to compress and accelerate ViT models while maintaining attention scores.
Snuffy: Efficient Whole Slide Image Classifier
Hossein Jafarinia (Sharif University of Technology), Mohammad Hossein Rohban (Sharif University of Technology)
ClassificationComputational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningBiomedical Data
🎯 What it does: Proposed a MIL-pooling architecture named Snuffy, which utilizes sparse Transformers for efficient classification of whole slide images (WSI), and employs self-supervised pre-training and Adapter for few-shot fine-tuning.
Soft Prompt Generation for Domain Generalization
Shuanghao Bai (Xi'an Jiaotong University), Badong Chen (Xi'an Jiaotong University)
Domain AdaptationRepresentation LearningTransformerPrompt EngineeringVision Language ModelGenerative Adversarial NetworkImageTextMultimodality
🎯 What it does: Propose a soft prompt generation method based on a generative model (SPG), which first learns domain-level soft prompt labels and then uses a conditional GAN (CGAN) to generate instance-level soft prompts for each sample, thereby improving the performance of vision-language models (e.g., CLIP) in domain generalization tasks.
Soft Shadow Diffusion (SSD): Physics-inspired Learning for 3D Computational Periscopy
Fadlullah A Raji (University of South Florida), John Murray-Bruce (University of South Florida)
GenerationData SynthesisOptimizationDiffusion modelImagePoint CloudMeshPhysics Related
🎯 What it does: By reformulating the light propagation model as a separable nonlinear least squares (SNLLS) problem, this paper achieves 3D structure reconstruction of hidden scenes from a single ordinary soft shadow photograph; simultaneously, two solution approaches are proposed: gradient-based alternating optimization and a generative neural network inspired by the physical model (Soft Shadow Diffusion, SSD). This network can directly generate 3D point clouds of occluding objects under given shadow conditions, and further locate and complete the 2D radiance maps of hidden objects, enabling simultaneous reconstruction of occluding objects' 3D shapes and non-occluded objects' 2D structures from a single photograph.
Solving Motion Planning Tasks with a Scalable Generative Model
Yihan Hu (Horizon Robotics Inc.), Qiang Liu (Horizon Robotics Inc.)
Autonomous DrivingRecurrent Neural NetworkTransformerWorld ModelSequential
🎯 What it does: Propose a unified motion planning framework called GUMP based on generative models, which can learn driving scene dynamics, generate diverse future trajectories, and construct new scenarios under different conditions;
Solving the inverse problem of microscopy deconvolution with a residual Beylkin-Coifman-Rokhlin neural network
Rui Li (Center for Advanced Systems Understanding), Artur Yakimovich (University of Wrocław)
RestorationConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: Proposed a physics-constrained residual Beylkin-Coifman-Rokhlin (m-rBCR) neural network to address the inverse problem of optical microscopy deconvolution.
SOS: Segment Object System for Open-World Instance Segmentation With Object Priors
Christian Wilms (University of Hamburg), Simone Frintrop (University of Hamburg)
SegmentationConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: Propose the Segment Object System (SOS), which first utilizes self-supervised ViT's self-attention maps as object priors to generate object-focused point prompts for the Segment Anything Model (SAM), obtaining high-quality pseudo annotations. These pseudo annotations are then mixed with real annotations of known categories to train a standard instance segmentation model, achieving open-world instance segmentation (OWIS).
Source Prompt Disentangled Inversion for Boosting Image Editability with Diffusion Models
Ruibin Li (Hong Kong Polytechnic University), Lei Zhang (Hong Kong Polytechnic University)
GenerationDiffusion modelImage
🎯 What it does: Design a source prompt decoupling inversion method called SPDInv to enhance the editability of text-driven image editing based on diffusion models.
Source-Free Domain-Invariant Performance Prediction
Ekaterina Khramtsova (University of Queensland), Mathieu Salzmann (École Polytechnique Fédérale de Lausanne)
Domain AdaptationScore-based ModelImage
🎯 What it does: Propose a domain-invariant performance prediction method that utilizes only the pre-trained model without relying on source data
SpaceJAM: a Lightweight and Regularization-free Method for Fast Joint Alignment of Images
Nir Barel (Ben Gurion University of Negev), Oren Freifeld (Ben Gurion University of Negev)
Auto EncoderContrastive LearningImage
🎯 What it does: Propose a lightweight, non-regularized image joint alignment method called SpaceJAM, which achieves fast and stable alignment by performing PCA+AE dimensionality reduction on DINO features, recursive STN with Lie group parameterization, and utilizing inverse composition loss.
SPAMming Labels: Efficient Annotations for the Trackers of Tomorrow
Orcun Cetintas (Technical University of Munich), Laura Leal-Taixé (NVIDIA)
Object TrackingData SynthesisConvolutional Neural NetworkGraph Neural NetworkVideo
🎯 What it does: Proposed a video labeling engine called SPAM, which combines synthetic pre-training, pseudo-labeling, active learning, and a graph structure hierarchical model to achieve high-quality annotation on multi-object tracking datasets while significantly reducing manual labor costs.
SPARO: Selective Attention for Robust and Compositional Transformer Encodings for Vision
Ankit Vani (Mila Universite De Montreal), Aaron Courville (Mila Universite De Montreal)
ClassificationRetrievalRepresentation LearningTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: Propose Sparo, replacing the Transformer's end to generate concept-divided slots, enhancing CLIP/DINO representation learning.
SpaRP: Fast 3D Object Reconstruction and Pose Estimation from Sparse Views
Chao Xu (UCLA), Minghua Liu (Hillbot Inc.)
GenerationPose EstimationConvolutional Neural NetworkMixture of ExpertsDiffusion modelImageMesh
🎯 What it does: Proposes the SpaRP method, which is capable of rapidly generating 3D textured meshes and estimating camera pose from a small number of 2D views without pose information.
Sparse Beats Dense: Rethinking Supervision in Radar-Camera Depth Completion
Huadong Li (MEGVII Technology), Renhe Ji (MEGVII Technology)
Depth EstimationAutonomous DrivingConvolutional Neural NetworkTransformerSupervised Fine-TuningImageMultimodalityPoint Cloud
🎯 What it does: Propose a Disruption-Compensation framework that reconstructs depth maps using sparse radar and camera data. By disrupting and compensating for the stripe-like scanning patterns (LiDAR Distribution Leakage, LDL) that emerge under sparse LiDAR supervision, the method significantly enhances depth prediction quality.
Sparse Refinement for Efficient High-Resolution Semantic Segmentation
Zhijian Liu, Song Han
SegmentationConvolutional Neural NetworkImage
🎯 What it does: Proposed SparseRefine, combining low-resolution full-image inference with sparse high-resolution pixel refinement to accelerate high-resolution semantic segmentation.
SparseCraft: Few-Shot Neural Reconstruction through Stereopsis Guided Geometric Linearization
Mae Younes (Inria), Adnane Boukhayma (Inria)
GenerationOptimizationNeural Radiance FieldImagePoint Cloud
🎯 What it does: Learn implicit SDF and radiance fields from a few color images to achieve 3D reconstruction and novel view synthesis under sparse viewpoints.
SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models
Yuwei Guo (Chinese University of Hong Kong), Bo Dai (Shanghai Artificial Intelligence Laboratory)
GenerationData SynthesisDiffusion modelImageVideoText
🎯 What it does: Add a sparse control encoder to a pre-trained text-to-video diffusion model, utilizing structural information from a few keyframes (such as sketches, depth, or RGB images) to guide video generation.
SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detection
Hongcheng Zhang (SenseTime Research), Zhe Wang (SenseTime Research)
Object DetectionMultimodalityPoint CloudBenchmark
🎯 What it does: Proposed a high-performance sparse multi-modal 3D object detector called SparseLIF, aiming to narrow the performance gap between sparse and dense detectors by enhancing the rich representations of LiDAR and camera data.
SparseRadNet: Sparse Perception Neural Network on Subsampled Radar Data
Jialong Wu (University of Wuppertal), Matthias Rottmann (Aptiv Services Deutschland GmbH)
Object DetectionSegmentationConvolutional Neural NetworkGraph Neural Network
🎯 What it does: Propose a sparse-aware network, SparseRadNet, which performs adaptive subsampling on raw radar data and efficiently extracts features through dual branches (GNN + SCNN) and attention fusion, achieving target detection and free space segmentation for sparse radar signals.
SparseSSP: 3D Subcellular Structure Prediction from Sparse-View Transmitted Light Images
Jintu Zheng (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences), Zenan Wang (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences)
GenerationConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: Developed the SparseSSP framework, which can predict 3D fluorescent cell structures under sparse perspective transmission images, significantly reducing acquisition frequency;
Spatial-Temporal Multi-level Association for Video Object Segmentation
Deshui Miao (Harbin Institute of Technology), Ming-Hsuan Yang (Dalian University of Technology)
SegmentationTransformerVideo
🎯 What it does: Proposes a semi-supervised video object segmentation framework based on spatial-temporal multi-layer association, achieving efficient association of object features and ID assignment through the STML module and spatial-temporal memory bank.
SpatialFormer: Towards Generalizable Vision Transformers with Explicit Spatial Understanding
Han Xiao (Tsinghua University), Jiwen Lu (Tsinghua University)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: Propose SpatialFormer, a decoder-only visual Transformer that achieves explicit spatial understanding of images through adaptive spatial tokens and bidirectional cross-attention, which can be directly transferred to multiple tasks such as classification, detection, and segmentation.
Spatially-Variant Degradation Model for Dataset-free Super-resolution
SHAOJIE GUO, Yan Wang (East China Normal University)
RestorationSuper ResolutionConvolutional Neural NetworkImageStochastic Differential Equation
🎯 What it does: Proposed a dataset-agnostic spatially varying degradation model for blind image super-resolution, capable of learning independent degradation kernels for each pixel in the image.