ECCV 2024 Papers — Page 19
European Conference on Computer Vision · 2387 papers
Revisit Event Generation Model: Self-Supervised Learning of Event-to-Video Reconstruction with Implicit Neural Representations
Zipeng Wang (Hong Kong University of Science and Technology), Lin Wang (Hong Kong University of Science and Technology)
GenerationVideoOrdinary Differential Equation
🎯 What it does: This paper proposes a self-supervised event-to-video reconstruction framework, EvINR, which directly utilizes the partial differential equations of event generation models to recover continuous intensity frames through implicit neural representations (INR), without requiring synthetic labels or optical flow estimation.
Revisit Human-Scene Interaction via Space Occupancy
Xinpeng Liu (Shanghai Jiao Tong University), Cewu Lu (Shanghai Jiao Tong University)
GenerationData SynthesisTransformerSupervised Fine-TuningSequential
🎯 What it does: By reconstructing human-scene interaction as spatial occupancy interaction, we constructed a Motion Occupancy Base (MOB) and trained an autoregressive occupancy space motion controller capable of handling dynamic environments.
Revisit Self-supervision with Local Structure-from-Motion
Shengjie Zhu (Michigan State University), Xiaoming Liu (Michigan State University)
Pose EstimationDepth EstimationAutonomous DrivingOptimizationConvolutional Neural NetworkNeural Radiance FieldVideoPoint Cloud
🎯 What it does: Propose a self-supervised depth estimation framework based on local SfM, utilizing depth and correspondence maps for Bundle-RANSAC-Adjustment pose optimization, and performing dense triangulation and geometric verification using a network-free Frustum Radiance Field.
Revisiting Adaptive Cellular Recognition Under Domain Shifts: A Contextual Correspondence View
Jianan Fan (University of Sydney), Weidong Cai (Microsoft)
SegmentationDomain AdaptationKnowledge DistillationConvolutional Neural NetworkTransformerGenerative Adversarial NetworkContrastive LearningImageBiomedical Data
🎯 What it does: This paper proposes a cross-domain cell nucleus recognition framework based on biological context correspondence, which learns high-level pathological generation principles from implicit correspondence relationships between cells and tissues, as well as between cells, through unsupervised self-supervised tasks, thereby achieving domain adaptation.
Revisiting Calibration of Wide-Angle Radially Symmetric Cameras
Andrea Porfiri Dal Cin (Politecnico di Milano), Luca Magri (Politecnico di Milano)
Autonomous DrivingOptimizationConvolutional Neural NetworkImage
🎯 What it does: Propose a two-step CNN-based radial symmetric camera calibration framework, first using a network to predict implicit camera representation VaCR, then solving parameters for arbitrary radial camera models through robust nonlinear optimization.
Revisiting Domain-Adaptive Object Detection in Adverse Weather by the Generation and Composition of High-Quality Pseudo-Labels
Rui Zhao (Shenzhen University), Shuoyao Wang (Shenzhen University)
Image TranslationRestorationObject DetectionSuper ResolutionDomain AdaptationImage
🎯 What it does: A generation-composition framework is constructed for cross-domain adverse weather object detection, with core components including IAoU loss improvement for regression, joint filtering combined with student perception for pseudo-label screening, and image enhancement based on rendering/recovery and super-resolution.
Revisiting Feature Disentanglement Strategy in Diffusion Training and Breaking Conditional Independence Assumption in Sampling
Wonwoong Cho (Adobe Research), Ajinkya Kale (Purdue University)
GenerationRepresentation LearningConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: This study proposes a training framework called FDiff and two sampling methods (GCDM and moment scheduling) to enhance the controllability and realism of diffusion models.
Revisiting Supervision for Continual Representation Learning
Daniel Marczak (IDEAS NCBR), Bartlomiej Twardowski
ClassificationRepresentation LearningSupervised Fine-TuningContrastive LearningImage
🎯 What it does: This paper investigates the impact of supervised information on representation learning in continual learning, and demonstrates that in the continual finetuning scenario, supervised learning models with an MLP projector (SL+MLP) can generate higher quality and more transferable features compared to self-supervised learning (SSL) models.
RGBD GS-ICP SLAM
Seongbo Ha (Sungkyunkwan University), Hyeonwoo Yu (Sungkyunkwan University)
Gaussian SplattingSimultaneous Localization and MappingImagePoint Cloud
🎯 What it does: Propose a real-time dense representation SLAM framework that integrates G-ICP with 3D Gaussian Splatting (3DGS), which simultaneously performs tracking and mapping on a single Gaussian map, achieving a high frame rate of 107 FPS and excellent map quality.
RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos
Tanveer Hannan (LMU Munich), Gedas Bertasius (LMU Munich)
RetrievalTransformerVision Language ModelContrastive LearningVideo
🎯 What it does: Proposes an RGNet, a unified long video text retrieval and localization network, achieving end-to-end retrieval and localization of specified events in 20-120 minute long videos.
RING-NeRF : Rethinking Inductive Biases for Versatile and Efficient Neural Fields
Doriand Petit (Université Paris-Saclay), Loïc Barthe (Université Paris-Saclay)
GenerationData SynthesisNeural Radiance FieldImage
🎯 What it does: Proposed a novel NeRF structure called RING-NeRF, leveraging continuous multi-scale residual grids and decoder spatial invariance to robustly address common challenges such as observation distance, few-view scenarios, SDF reconstruction, and supports dynamic resolution scaling.
Risk-Aware Self-Consistent Imitation Learning for Trajectory Planning in Autonomous Driving
Yixuan Fan (Tsinghua University), Shengjin Wang (Tsinghua University)
Autonomous DrivingRecurrent Neural NetworkTransformerPoint CloudSequential
🎯 What it does: In autonomous driving trajectory planning, the risk-aware and self-consistent imitation learning framework RaSc is proposed, which learns human drivers' risk awareness through predictions of time-to-collision (TTC), and enables the model to understand the consequences of its own actions via self-consistency constraints.
RISurConv: Rotation Invariant Surface Attention-Augmented Convolutions for 3D Point Cloud Classification and Segmentation
Zhiyuan Zhang (Singapore Management University), Zhiyu Xiang (Zhejiang University)
ClassificationSegmentationConvolutional Neural NetworkTransformerPoint Cloud
🎯 What it does: Propose a rotation-invariant surface attention-enhanced convolution (RISurConv) based on local triangular face construction for 3D point cloud classification and segmentation tasks.
RoadPainter: Points Are Ideal Navigators for Topology transformER
Zhongxing Ma (Baidu), Guowei Wan (Baidu)
Object DetectionSegmentationAutonomous DrivingConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes the RoadPainter method, which first regresses lane centerline points in the BEV space from multi-view images, and then refines them using point-guided masks to complete centerline instance prediction and topological reasoning.
Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation
Yuanchen Ju (Shanghai Qi Zhi Institute), Huazhe Xu (Tsinghua University)
Robotic IntelligenceDiffusion modelImageVideo
🎯 What it does: By extracting hand-object interaction experiences from human-perspective videos, constructing robot interaction memory, and leveraging the semantic correspondence capability of diffusion models to map contact points of known objects to unknown objects, achieving cross-category zero-shot grasping;
Robust Calibration of Large Vision-Language Adapters
Balamurali Murugesan (ETS Montreal), Jose Dolz (ETS Montreal)
Domain AdaptationRepresentation LearningTransformerPrompt EngineeringContrastive LearningImageText
🎯 What it does: This paper investigates adaptation methods for CLIP in out-of-distribution (OOD) scenarios (Prompt Learning, Adapters, Test-Time Prompt Tuning) and finds that while they improve accuracy, they severely lose calibration. To address this issue, the authors propose three calibration strategies based on logit range normalization (ZS-Norm, Penalty, SaLS).
Robust Fitting on a Gate Quantum Computer
Frances F Yang, Tat-Jun Chin (University of Adelaide)
Autonomous DrivingOptimizationPoint CloudBenchmarkPhysics Related
🎯 What it does: Implemented and demonstrated the first complete workflow for robust fitting on a gate-based quantum computer (IonQ Aria), with the core being the proposal and implementation of a quantum ℓ∞ feasibility test subcircuit for 1D point fitting, and achieving robust estimation of higher-dimensional nonlinear models (e.g., baseline matrices) through the accumulation of 1D Boolean influences.
Robust Incremental Structure-from-Motion with Hybrid Features
Shaohui Liu (ETH Zurich), Marc Pollefeys (Microsoft Mixed Reality & AI Lab)
Pose EstimationOptimizationSimultaneous Localization and MappingImageBenchmark
🎯 What it does: Developed an end-to-end incremental Structure-from-Motion (SfM) system that integrates points, lines, vanishing points (VP), and their geometric relationships, achieving a complete workflow from feature detection, triangulation, registration to bundle adjustment;
Robust Multimodal Learning via Representation Decoupling
Shicai Wei (University of Electronic Science and Technology of China), Chunbo Luo (University of Electronic Science and Technology of China)
ClassificationRecognitionConvolutional Neural NetworkImageVideoMultimodalityAudio
🎯 What it does: Propose DMRNet, which models representations of different modality combinations as probability distributions and decouples training and inference representations, thereby alleviating the intra-class directional constraints of traditional subspace methods to achieve robust multi-modal learning;
Robust Nearest Neighbors for Source-Free Domain Adaptation under Class Distribution Shift
Antonio Tejero-de-Pablos (CyberAgent), Shin'ichi Satoh (National Institute of Informatics)
Domain AdaptationConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: Propose a source-agnostic domain adaptation method based on robust nearest neighbors, utilizing generic features to refine pseudo-labels and address class distribution shift problems.
Robust Zero-Shot Crowd Counting and Localization with Adaptive Resolution SAM
Jia Wan (Harbin Institute of Technology), Antoni Chan (City University of Hong Kong)
Object DetectionSegmentationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a zero-shot (unsupervised) crowd counting and localization method, which utilizes an adaptive resolution SEEM to generate pseudo labels, combines Gaussian Mixture Model (GMM) for head localization, and trains a counting network with a robust loss function, followed by iterative pseudo label optimization.
Robust-Wide: Robust Watermarking against Instruction-driven Image Editing
Runyi Hu (Nanyang Technological University), Tianwei Zhang (National University of Singapore)
GenerationConvolutional Neural NetworkVision Language ModelDiffusion modelAuto EncoderImageText
🎯 What it does: Developed a robust watermarking method called Robust-Wide for instruction-driven image editing.
Robustness Preserving Fine-tuning using Neuron Importance
Guangrui Li (Amazon AI), Jonathan Wu (Amazon AI)
Domain AdaptationExplainability and InterpretabilityRepresentation LearningSupervised Fine-TuningVision Language ModelContrastive LearningImage
🎯 What it does: This paper proposes a robust fine-tuning framework based on neuronal importance, which quantifies the importance of each neuron and uses this importance to guide selective fine-tuning or regularization, thereby improving the generalization ability of vision-language models in downstream tasks.
Robustness Tokens: Towards Adversarial Robustness of Transformers
Brian Pulfer (University of Geneva), Slava Voloshynovskiy (University of Geneva)
ClassificationSegmentationAdversarial AttackTransformerImage
🎯 What it does: Enhancing model robustness against adversarial attacks by learning a small number of private additional tokens (Robustness Tokens) on the Transformer architecture, while maintaining downstream task performance.
RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models
Bowen Zhang (University of Science and Technology of China), Baining Guo (Microsoft Research Asia)
GenerationDiffusion modelAuto EncoderImageMesh
🎯 What it does: This paper proposes RodinHD, which generates high-fidelity 3D avatars using diffusion models, avoiding detail loss and cross-view consistency issues in traditional methods.
RoDUS: Robust Decomposition of Static and Dynamic Elements in Urban Scenes
Thang-Anh-Quan Nguyen (Huawei Paris Research Center), Dzmitry Tsishkou (Huawei Paris Research Center)
SegmentationAutonomous DrivingNeural Radiance FieldVideoPoint Cloud
🎯 What it does: Propose the RoDUS network, which utilizes a dual-branch NeRF to decompose static and dynamic components in urban dynamic scenes under self-supervised learning conditions;
RoGUENeRF: A Robust Geometry-Consistent Universal Enhancer for NeRF
Sibi Catley-Chandar (Huawei Noah's Ark Lab), Eduardo Pérez Pellitero (Queen Mary University of London)
GenerationPose EstimationSuper ResolutionConvolutional Neural NetworkTransformerSupervised Fine-TuningNeural Radiance FieldOptical FlowImage
🎯 What it does: Propose RoGUENeRF, a NeRF post-processing enhancer based on 3D alignment, non-rigid refinement, and geometric attention, significantly improving rendering quality.
RoofDiffusion: Constructing Roofs from Severely Corrupted Point Data via Diffusion
Kyle Shih-Huang Lo (University of Florida), Eric Spellman (Meta Platforms Inc)
RestorationDiffusion modelPoint Cloud
🎯 What it does: Propose the RoofDiffusion method to achieve complete and denoised roof height map reconstruction under severely sparse, missing, and noisy roof height maps.
RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting
Qi WANG, Dan Xu (Hong Kong University of Science and Technology)
GenerationDiffusion modelTextPoint CloudMesh
🎯 What it does: Proposes a coarse-to-fine indoor scene texturization framework called RoomTex, which can generate high-quality, style-consistent textures for compositional 3D scenes under given text prompts, and supports interactive fine-tuning and scene editing.
RoScenes: A Large-scale Multi-view 3D Dataset for Roadside Perception
Xiaosu Zhu (Alibaba Cloud), Jieping Ye (Alibaba Cloud)
Autonomous DrivingTransformerImageMultimodalityPoint CloudBenchmark
🎯 What it does: Proposed a large-scale multi-perspective road-side 3D perception dataset named RoScenes, and designed an efficient BEV-to-3D joint annotation pipeline; to address challenges in RoScenes, introduced the RoBEV method, which achieves efficient 2D-3D feature assignment through feature-guided 3D position embedding, thereby enhancing BEV detection performance.
Rotary Position Embedding for Vision Transformer
Byeongho Heo (NAVER AI Lab), Sangdoo Yun (NAVER AI Lab)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: This paper studies and verifies the application of Rotating Position Embedding (RoPE) in Vision Transformer (ViT), and proposes axial and mixed learning frequency implementations for 2D RoPE.
Rotated Orthographic Projection for Self-Supervised 3D Human Pose Estimation
YAO YAO, Jiamao Li (Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences)
Pose EstimationGraph Neural NetworkContrastive LearningVideo
🎯 What it does: Proposed a self-supervised 3D human pose estimation framework that improves projection consistency through rotational orthogonal projection and joint reversal constraints.
RPBG: Towards Robust Neural Point-based Graphics in the Wild
Qingtian Zhu (University of Tokyo), Yinqiang Zheng (XREAL)
GenerationConvolutional Neural NetworkNeural Radiance FieldImagePoint CloudBenchmark
🎯 What it does: Proposes a robust point-based neural rendering framework, RPBG, for achieving high-quality view synthesis in diverse outdoor scenarios.
RS-NeRF: Neural Radiance Fields from Rolling Shutter Images
Muyao Niu (University of Tokyo), Yinqiang Zheng (University of Tokyo)
RestorationGenerationPose EstimationConvolutional Neural NetworkNeural Radiance FieldGaussian SplattingOptical FlowImage
🎯 What it does: Studied a method to directly learn NeRF from rolling shutter images, jointly optimizing camera trajectory and voxel networks to achieve rolling shutter distortion compensation and novel view synthesis.
RSL-BA: Rolling Shutter Line Bundle Adjustment
Yongcong Zhang (Hunan University), Yizhen Lao (Hunan University)
Pose EstimationOptimizationImage
🎯 What it does: This paper proposes the first rolling shutter bundle adjustment (RSL-BA) framework based on line features, deriving the curve projection under rolling shutter using Pucker line parameterization and constructing a stable projection error;
RT-Pose: A 4D Radar-Tensor based 3D Human Pose Estimation and Localization Benchmark
Yuan-Hao Ho (National Cheng Kung University), Jenq-Neng Hwang (National Cheng Kung University)
Pose EstimationConvolutional Neural NetworkImageMultimodalityPoint CloudBenchmark
🎯 What it does: Constructed the RT-Pose dataset and proposed the HRRadarPose single-stage radar tensor 3D human pose estimation model.
S-JEPA: A Joint Embedding Predictive Architecture for Skeletal Action Recognition
Mohamed Abdelfattah (EPFL), Alexandre Alahi (EPFL)
RecognitionTransformerAuto EncoderGraphTime Series
🎯 What it does: This paper proposes a self-supervised skeletal action recognition pre-training framework named S-JEPA, which utilizes partial skeletal sequences to predict the potential representations of missing joints;
S^3D-NeRF: Single-Shot Speech-Driven Neural Radiance Field for High Fidelity Talking Head Synthesis
Dongze Li (School of Artificial Intelligence, University of Chinese Academy of Sciences), Jing Dong (NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences)
GenerationData SynthesisSuper ResolutionConvolutional Neural NetworkTransformerNeural Radiance FieldGenerative Adversarial NetworkContrastive LearningImageVideoMultimodalityAudio
🎯 What it does: This study proposes a high-fidelity talking head synthesis method driven by a single photo and voice, capable of generating natural and audio-synchronized speaking videos from any perspective.
SA-DVAE: Improving Zero-Shot Skeleton-Based Action Recognition by Disentangled Variational Autoencoders
Sheng-Wei Li (National Taiwan University), Jane Yung-jen Hsu (National Taiwan University)
RecognitionAuto EncoderGraph
🎯 What it does: This paper proposes the SA-DVAE model, which improves zero-shot recognition of skeletal actions through feature separation and alignment.
SAFARI: Adaptive Sequence Transformer for Weakly Supervised Referring Expression Segmentation
Sayan Nag (University of Toronto), Srikrishna Karanam (Adobe Research)
SegmentationTransformerVision Language ModelImageVideoTextMultimodality
🎯 What it does: Propose a weakly supervised finger expression segmentation model, SafarI, which achieves high-quality segmentation using only 30% masks and bounding box annotations by leveraging autoregressive contour prediction.
Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models
Samuele Poppi (University of Modena and Reggio Emilia), Rita Cucchiara (University of Modena and Reggio Emilia)
GenerationData SynthesisRetrievalSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelDiffusion modelContrastive LearningMultimodality
🎯 What it does: Designed and implemented Safe-CLIP, which fine-tunes the text and visual encoders of CLIP to be insensitive to NSFW content in cross-modal retrieval, text-to-image, and image-to-text generation tasks.
Safe-Sim: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries
Wei-Jer Chang (UC Berkeley NEC Labs America), Manmohan Chandraker (UC San Diego)
Autonomous DrivingDiffusion modelPoint Cloud
🎯 What it does: Proposes the Safe-Sim framework, leveraging diffusion models to achieve controllable, closed-loop safety-critical traffic simulation, generating realistic long-tail collision scenarios and adjustable adversarial challenges against opponents.
Safeguard Text-to-Image Diffusion Models with Human Feedback Inversion
Sanghyun Kim (Korea Advanced Institute of Science and Technology), Juho Lee (Korea Advanced Institute of Science and Technology)
GenerationSafty and PrivacyReinforcement LearningPrompt EngineeringDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose the Human Feedback Inversion (HFI) framework, which compresses human feedback into soft word embeddings to eliminate harmful or copyright concepts in text-to-image diffusion models, and combines it with Safe Self-Distillation Diffusion (SDD) to further fine-tune the model and enhance concept removal effectiveness.
SAFNet: Selective Alignment Fusion Network for Efficient HDR Imaging
Lingtong Kong (vivo Mobile Communication Co., Ltd), Jinwei Chen (vivo Mobile Communication Co., Ltd)
RestorationConvolutional Neural NetworkOptical FlowImage
🎯 What it does: Propose a multi-exposure HDR image reconstruction network named SAFNet, which achieves fast ghost removal and high-quality HDR synthesis through selective alignment and explicit fusion.
SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning
Bac Nguyen (Sony Ai), Aaron Courville (Cifar Ai Chair)
ClassificationDomain AdaptationComputational EfficiencyRepresentation LearningSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: On the pre-trained vision-language model CLIP, SAFT achieves sparse fine-tuning by only updating a small portion of parameters with high gradient importance, avoiding performance degradation on out-of-distribution (OOD) data caused by traditional full fine-tuning.
SAGS: Structure-Aware 3D Gaussian Splatting
Evangelos Ververas (Imperial College London), Stefanos Zafeiriou (Imperial College London)
GenerationOptimizationComputational EfficiencyGraph Neural NetworkNeural Radiance FieldGaussian SplattingPoint CloudMesh
🎯 What it does: Proposed a structure-aware 3D Gaussian Splatting method (SAGS), which encodes point clouds via graph neural networks and learns point displacement and attributes while maintaining the scene's geometric structure;
SAH-SCI: Self-Supervised Adapter for Efficient Hyperspectral Snapshot Compressive Imaging
Haijin Zeng (IMEC-Ghent University), Jingyong Su (Harbin Institute of Technology (Shenzhen))
CompressionConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a self-supervised adapter (SAH), achieving unsupervised fine-tuning for hyperspectral snapshot compression imaging by adding a lightweight adaptation module after a frozen pre-trained model.
SAIR: Learning Semantic-aware Implicit Representation
Canyu Zhang (University of South Carolina), Song Wang (University of South Carolina)
RestorationConvolutional Neural NetworkVision Language ModelImage
🎯 What it does: This paper proposes a Semantic-Aware Implicit Representation (SAIR) framework to achieve continuous semantic and appearance mapping in image restoration tasks, thereby better reconstructing image content in large missing regions.
Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training
Hyesong Choi (Ewha W. University), Dongbo Min (New York University)
Representation LearningTransformerAuto EncoderImage
🎯 What it does: Proposes the Salience-Based Adaptive Masking (SBAM) method, which intelligently selects mask positions by calculating the salience of image tokens and achieves adaptive pre-training for each image through dynamic mask ratio (AMR).
SAM-COD: SAM-guided Unified Framework for Weakly-Supervised Camouflaged Object Detection
Huafeng Chen (Northwestern Polytechnical University), Shan Gao (Northwestern Polytechnical University)
Object DetectionKnowledge DistillationTransformerPrompt EngineeringImageBenchmark
🎯 What it does: Proposes a unified weakly supervised hidden object detection framework SAM-COD, supporting point, box, and sketch labels, leveraging SAM's prompt learning, response filtering, semantic matching, and prompt adaptive knowledge distillation to achieve high-quality pseudo labels and feature learning.
SAM-guided Graph Cut for 3D Instance Segmentation
Haoyu Guo (Zhejiang University), Xiaowei Zhou
SegmentationGraph Neural NetworkVision Language ModelImagePoint Cloud
🎯 What it does: This paper proposes a 3D-to-2D query framework that first oversegments 3D point clouds into superpoints, then uses SAM (Segment Anything Model) to generate segmentation masks from multiple perspectives to construct node features and edge weights of the superpoint graph, and finally employs a graph neural network (GNN) to learn edge similarity and perform graph cutting, achieving high-quality 3D instance segmentation results.
SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation
Yi-Chia Chen (National Taiwan University), Chu-Song Chen (National Taiwan University)
SegmentationLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: Integrate the Segment Anything Model (SAM) with a multi-modal large language model (MLLM) to construct SAM4MLLM, achieving pixel-level learning for referential expression segmentation (RES) tasks without modifying the MLLM architecture or adding dedicated tokens.
SAMFusion: Sensor-Adaptive Multimodal Fusion for 3D Object Detection in Adverse Weather
Edoardo Palladin (Torc Robotics), Felix Heide (Torc Robotics)
Object DetectionAutonomous DrivingTransformerImageMultimodalityPoint Cloud
🎯 What it does: Developed a multi-modal perception framework named SAMFusion for 3D object detection under adverse weather conditions by fusing infrared gated camera, RGB, LiDAR, and millimeter-wave radar.
Sapiens: Foundation for Human Vision Models
Rawal Khirodkar (Codec Avatars Lab, Meta), Shunsuke Saito (Codec Avatars Lab, Meta)
SegmentationPose EstimationDepth EstimationRepresentation LearningTransformerAuto EncoderImage
🎯 What it does: Developed a series of Sapiens vision Transformer models, pre-trained and fine-tuned for four human-centric tasks: pose estimation, human segmentation, depth estimation, and surface normal prediction.
SAVE: Protagonist Diversification with Structure Agnostic Video Editing
Yeji Song (Seoul National University), Nojun Kwak (Seoul National University)
GenerationConvolutional Neural NetworkDiffusion modelOptical FlowVideoBenchmark
🎯 What it does: Propose an editing method (SAVE) that allows free replacement of the main subject in videos while maintaining motion, addressing the issues of existing methods' localization bias for motion-related terms and difficulty in maintaining motion when the subject's shape changes.
SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer
Zijie Wu (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
GenerationData SynthesisDiffusion modelScore-based ModelGaussian SplattingVideoMesh
🎯 What it does: Proposes the SC4D framework to generate high-quality 4D dynamic 3D models from single-view videos and supports motion transfer.
Scalable Group Choreography via Variational Phase Manifold Learning
Nhat Le (AIOZ), Anh Nguyen (University of Liverpool)
GenerationTransformerAuto EncoderVideo
🎯 What it does: Developed a scalable group dance generation model based on a phase variational autoencoder, capable of generating synchronized dances with any number of dancers under fixed GPU memory.
Scalar Function Topology Divergence: Comparing Topology of 3D Objects
Ilya Trofimov (Skolkovo Institute of Science and Technology), Serguei Barannikov (Skolkovo Institute of Science and Technology)
SegmentationMeshGraphBiomedical Data
🎯 What it does: Proposed and implemented a new topology comparison tool called Scalar Function Topology Divergence (SFTD), along with its corresponding F-Cross-Barcode, to measure and locate multi-scale topological differences between sublevel sets of two scalar functions, and applied SFTD as a loss function for 3D shape reconstruction and segmentation tasks.
ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillation
Zhiyuan Ma (Hong Kong Polytechnic University), Lei Zhang (Hong Kong Polytechnic University)
Data SynthesisConvolutional Neural NetworkDiffusion modelScore-based ModelTextMesh
🎯 What it does: Propose Asynchronous Score Distillation (ASD), achieving scalable text-to-3D synthesis by reducing noise prediction error through time-step advancement without fine-tuning diffusion model weights.
Scaling Backwards: Minimal Synthetic Pre-training?
Ryo Nakamura (National Institute of Advanced Industrial Science and Technology), Hirokatsu Kataoka (National Institute of Advanced Industrial Science and Technology)
ClassificationData SynthesisTransformerSupervised Fine-TuningImage
🎯 What it does: Propose using a single fractal image with noise transformation to construct a minimal synthetic pre-training dataset called 1p-frac in visual pre-training, and achieve model pre-training through LPCE loss.
Scaling Up Personalized Image Aesthetic Assessment via Task Vector Customization
Jooyeol Yun (Korea Advanced Institute of Science and Technology), Jaegul Choo (Korea Advanced Institute of Science and Technology)
TransformerSupervised Fine-TuningImage
🎯 What it does: Integrate task vectors from multiple general image aesthetics and image quality databases to perform precise personalized assessment of individual aesthetic preferences using a small number of user samples.
ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilities
Chenming Zhu (The University of Hong Kong), Xihui Liu (Shanghai AI Laboratory)
TransformerLarge Language ModelSupervised Fine-TuningVision Language ModelTextPoint CloudBenchmarkChain-of-Thought
🎯 What it does: Propose the 3D reasoning localization task and the ScanReason benchmark, and design the ReGround3D framework, which combines a vision-centric reasoning module with a geometry-enhanced backtracking 3D localization module to achieve implicit instruction reasoning and localization.
ScanTalk: 3D Talking Heads from Unregistered Scans
Federico Nocentini (University of Florence), Mohamed Daoudi (IMT Nord Europe)
GenerationRecurrent Neural NetworkGraph Neural NetworkDiffusion modelMeshAudio
🎯 What it does: This paper proposes a deep learning framework called ScanTalk, which can generate animations of arbitrary topology 3D facial meshes driven by speech, without requiring pre-registration of the mesh.
SCAPE: A Simple and Strong Category-Agnostic Pose Estimator
Yujia Liang (Huazhong University of Science and Technology), Hao Lu (Huazhong University of Science and Technology)
Pose EstimationTransformerImage
🎯 What it does: Propose the SCAPE model, adopting a simplified architecture with pure self-attention layers and an MLP regression head to directly perform category-agnostic pose estimation.
ScatterFormer: Efficient Voxel Transformer with Scattered Linear Attention
Chenhang He (Hong Kong Polytechnic University), Lei Zhang (Hong Kong Polytechnic University)
Object DetectionAutonomous DrivingTransformerPoint Cloud
🎯 What it does: Propose ScatterFormer, a voxel transformer utilizing hash-based linear attention (SLA) and cross-window interaction (CWI) for large-scale point cloud 3D detection, significantly reducing computational and memory overhead under sparse point clouds.
Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizer
Eric Brachmann (Niantic), Victor Adrian Prisacariu (Niantic)
Pose EstimationSimultaneous Localization and MappingImage
🎯 What it does: Proposes an incremental learning SfM framework ACE0 based on scene coordinate regression, capable of estimating camera poses from a set of RGB images without initial poses.
Scene-aware Human Motion Forecasting via Mutual Distance Prediction
Chaoyue Xing (Australian National University), Miaomiao Liu (Australian National University)
Pose EstimationRecurrent Neural NetworkGraph Neural NetworkSequential
🎯 What it does: Propose a scene-aware human 3D motion prediction method through mutual distance prediction;
Scene-Conditional 3D Object Stylization and Composition
Jinghao Zhou (University of Oxford), Christian Rupprecht (University of Oxford)
GenerationData SynthesisLarge Language ModelVision Language ModelDiffusion modelImageTextMesh
🎯 What it does: This paper proposes a framework that adapts the texture and lighting of existing 3D assets to a specified 2D scene, achieving seamless and realistic synthesis within the scene, and ultimately generating 3D models directly applicable to downstream tasks such as games.
Scene-Graph ViT: End-to-End Open-Vocabulary Visual Relationship Detection
Tim Salzmann (Google DeepMind), Matthias Minderer (Google DeepMind)
RecognitionRetrievalTransformerVision Language ModelImageMultimodality
🎯 What it does: Propose a Transformer-based encoder-only architecture that utilizes a relation attention mechanism to achieve open-vocabulary visual relation detection, enabling end-to-end training.
SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs
Yang Miao (ETH Zurich), Daniel Barath
RetrievalCompressionKnowledge DistillationRepresentation LearningConvolutional Neural NetworkGraph Neural NetworkTransformerContrastive LearningImageMultimodalityPoint CloudGraph
🎯 What it does: Propose a cross-modal coarse localization method called SceneGraphLoc based on 3D scene graphs, which can locate query images in a database storing only 3D scene graphs
SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model
Armen Avetisyan (Meta Reality Labs), Vasileios Balntas (Meta Reality Labs)
GenerationTransformerLarge Language ModelVideoTextPoint Cloud
🎯 What it does: Directly converting first-person video into complete 3D scene descriptions (walls, doors, windows, and 3D bounding boxes) through an autoregressive structured language model
SceneTeller: Language-to-3D Scene Generation
Basak Melis Ocal, Theo Gevers (Robert Bosch GmbH)
GenerationData SynthesisRetrievalTransformerLarge Language ModelGaussian SplattingTextMeshRetrieval-Augmented Generation
🎯 What it does: Propose SceneTeller, an end-to-end pipeline that generates complete 3D room scenes from natural language descriptions and supports style editing for scenes or individual objects.
SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding
Baoxiong Jia (State Key Laboratory of General Artificial Intelligence), Siyuan Huang (State Key Laboratory of General Artificial Intelligence)
TransformerLarge Language ModelVision Language ModelContrastive LearningTextMultimodalityPoint Cloud
🎯 What it does: Constructed a large-scale 3D vision-language dataset named SceneVerse and proposed a unified multi-level alignment pre-training framework GPS to enhance 3D scene language alignment and downstream task performance.
Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks
Jing Wu (Monash University), Mehrtash Harandi (Monash University)
OptimizationSafty and PrivacyComputational EfficiencyData-Centric LearningImageMultimodality
🎯 What it does: Propose a novel machine unlearning method called Scissorhands, which can effectively remove the impact of specified data while maintaining the model's performance on the remaining data.
SCLIP: Rethinking Self-Attention for Dense Vision-Language Inference
Feng Wang (Johns Hopkins University), Alan Yuille (Johns Hopkins University)
SegmentationTransformerPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: By improving CLIP's self-attention mechanism, the SCLIP model is proposed, achieving open-source semantic segmentation without additional training.
SCOD: From Heuristics to Theory
Vojtech Franc (Czech Technical University in Prague), Daniel Prusa (Czech Technical University in Prague)
ClassificationAnomaly DetectionImageBenchmark
🎯 What it does: This paper studies how to design a reliable selective classifier in environments containing OOD samples, and clarifies the optimal strategy and implementation methods.
SCOMatch: Alleviating Overtrusting in Open-set Semi-supervised Learning
Zerun Wang (University of Tokyo), Toshihiko Yamasaki (University of Tokyo)
ClassificationImageBenchmark
🎯 What it does: To address the problem of overfitting on the decision boundary caused by excessive trust in labeled ID samples in open-ended semi-supervised learning, the SCOMatch method is proposed, treating OOD as an additional class to construct a (K+1)-class SSL.
Score Distillation Sampling with Learned Manifold Corrective
Thiemo Alldieck (Google Research), Cristian Sminchisescu (Google Research)
Image TranslationGenerationConvolutional Neural NetworkDiffusion modelScore-based ModelImageText
🎯 What it does: This paper conducts an in-depth analysis of the Score Distillation Sampling (SDS) loss, identifying issues such as noise and over-saturation caused by frequency bias in its gradients. It proposes a learning-based manifold correction (LMC-SDS) to eliminate this bias, thereby obtaining more stable and higher-quality gradients. The effectiveness of this method is subsequently validated across multiple tasks, including image synthesis, image editing, training of zero-shot image translation networks, and 3D asset generation.
SCP-Diff: Spatial-Categorical Joint Prior for Diffusion Based Semantic Image Synthesis
Huan-ang Gao (Tsinghua University), Hao Zhao
GenerationData SynthesisDiffusion modelAuto EncoderImage
🎯 What it does: Proposed and implemented a spatial-category joint noise prior (SCP-Diff) for semantic image synthesis, addressing the issues of singular substructures and semantic shifts caused by mismatched noise distributions between training and inference phases in traditional ControlNet, by utilizing precomputed spatial, category, and joint priors during inference.
SCPNet: Unsupervised Cross-modal Homography Estimation via Intra-modal Self-supervised Learning
Runmin Zhang (Zhejiang University), Si-Yuan Cao (Zhejiang University)
Convolutional Neural NetworkImageMultimodality
🎯 What it does: Propose an unsupervised cross-modal homography estimation framework SCPNet, achieving precise alignment of cross-modal images such as satellite images and map images through internal self-supervised learning, correlation calculation, and consistent feature mapping projection.
ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image
Hallee E. Wong (MIT), Adrian V. Dalca (MIT)
SegmentationConvolutional Neural NetworkTransformerMultimodalityBiomedical Data
🎯 What it does: Proposed ScribblePrompt, a neural network-based interactive segmentation tool that supports multiple interaction methods such as sketching, clicking, and framing, enabling rapid and accurate segmentation on unseen medical imaging tasks.
SDPT: Synchronous Dual Prompt Tuning for Fusion-based Visual-Language Pre-trained Models
Yang Zhou (Beihang University), Yan Xu (Zhejiang University)
Object DetectionTransformerPrompt EngineeringVision Language ModelMultimodality
🎯 What it does: This paper proposes a synchronous dual-modal prompt tuning method called SDPT for fused vision-language pre-training models (e.g., GLIP), aiming to achieve parameter-efficient fine-tuning for downstream tasks.
SeA: Semantic Adversarial Augmentation for Last Layer Features from Unsupervised Representation Learning
Qi Qian (Alibaba Group), Juhua Hu (University of Washington)
ClassificationRepresentation LearningAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: Propose a semantic adversarial enhancement (SeA) method in a fixed-depth feature space, generating semantically rich perturbations by projecting gradients into the real sample subspace, and training a linear classifier using enhanced features.
SEDiff: Structure Extraction for Domain Adaptive Depth Estimation via Denoising Diffusion Models
Dongseok Shim (Seoul National University), Hyoun Jin Kim
Depth EstimationDomain AdaptationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: Designed and implemented a single-view depth estimation framework called SEDiff, which utilizes diffusion models for domain-agnostic structure extraction and depth-consistent style transfer to address the adaptation problem from synthetic to real domains.
See and Think: Embodied Agent in Virtual Environment
Zhonghan Zhao (Zhejiang University), Gaoang Wang (Zhejiang University)
Robotic IntelligenceConvolutional Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningVision-Language-Action ModelImageTextMultimodalityRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes STEVE, a full-modal voxel agent for Minecraft that can accomplish complex tasks through an integrated approach of visual perception, LLM-driven language instructions, and code actions.
SEED: A Simple and Effective 3D DETR in Point Clouds
Zhe Liu (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
Object DetectionAutonomous DrivingTransformerPoint Cloud
🎯 What it does: Propose a 3D object detection framework SEED based on DETR, achieving high-quality queries and efficient feature interaction on point clouds through Dual Query Selection (DQS) and Deformable Grid Attention (DGA).
Seeing Faces in Things: A Model and Dataset for Pareidolia
Mark T Hamilton (MIT), William T. Freeman (MIT)
Object DetectionConvolutional Neural NetworkSupervised Fine-TuningImageBenchmark
🎯 What it does: This paper constructs a large-scale 'Faces in Things' face illusion dataset to evaluate the performance of existing face detectors in identifying illusory faces, and explores how models can generate similar illusions through data augmentation and transfer learning.
Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration
Shihao Zhou (Nankai University), Jufeng Yang (Nankai University)
RestorationTransformerPrompt EngineeringImage
🎯 What it does: Propose a frequency-prompt-based Transformer model called FPro, which decomposes features into low-frequency and high-frequency components through gated dynamic decoupling, and uses dual-frequency prompt modules to separately generate and modulate prompts for image restoration;
SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving
Qingwen Zhang (KTH Royal Institute of Technology), Patric Jensfelt (KTH Royal Institute of Technology)
Autonomous DrivingRecurrent Neural NetworkOptical FlowPoint Cloud
🎯 What it does: Propose a self-supervised scene flow estimation method called SeFlow, which improves the accuracy of LiDAR point cloud scene flow by utilizing dynamic point classification and clustering consistency constraints.
SegGen: Supercharging Segmentation Models with Text2Mask and Mask2Img Synthesis
Hanrong Ye (HKUST), Dan Xu (Adobe Research)
SegmentationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelDiffusion modelImageText
🎯 What it does: Propose SegGen, which generates diverse segmentation masks (Text2Mask) from text, and then generates realistic images (Mask2Img) from masks, thereby synthesizing large-scale high-quality segmentation training data without relying on segmentation labelers.
SEGIC: Unleashing the Emergent Correspondence for In-Context Segmentation
Lingchen Meng (Fudan University), Yu-Gang Jiang (Fudan University)
SegmentationTransformerPrompt EngineeringContrastive LearningImageVideo
🎯 What it does: Propose SegIC, an end-to-end context segmentation framework based on vision foundation models, achieving segmentation from few examples to target images by leveraging dense correspondence relationships.
Segment and Recognize Anything at Any Granularity
Feng Li (Hong Kong University of Science and Technology), Jianfeng Gao (Microsoft Research)
RecognitionSegmentationTransformerImage
🎯 What it does: This paper proposes Semantic-SAM, a universal segmentation model capable of performing semantic recognition and segmentation at any granularity.
Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without Manual Labels
Rui Huang, Francis Engelmann
SegmentationDepth EstimationConvolutional Neural NetworkTransformerSupervised Fine-TuningImagePoint Cloud
🎯 What it does: This paper proposes Segment3D, a fine-grained, class-agnostic 3D point cloud segmentation method that does not require manual annotation.
Segmentation-guided Layer-wise Image Vectorization with Gradient Fills
Hengyu Zhou (Tsinghua University), Bin Wang (Tsinghua University)
Image TranslationImage
🎯 What it does: Propose a segmentation-guided hierarchical vectorization framework that can convert raster images into vector graphics with radial gradient fills;
SegPoint: Segment Any Point Cloud via Large Language Model
Shuting He (Nanyang Technological University), Bihan Wen (Nanyang Technological University)
SegmentationConvolutional Neural NetworkTransformerLarge Language ModelVision Language ModelMultimodalityPoint Cloud
🎯 What it does: Proposed SegPoint, a unified framework for 3D point cloud segmentation using multi-modal LLM, and created the Instruct3D dataset
SegVG: Transferring Object Bounding Box to Segmentation for Visual Grounding
Weitai Kang (Illinois Institute of Technology), Yan Yan (Cisco Research)
RecognitionSegmentationTransformerVision Language ModelMultimodality
🎯 What it does: Propose the SegVG method, which converts bounding box annotations into pixel-level segmentation supervision to improve visual localization tasks.
SeiT++: Masked Token Modeling Improves Storage-efficient Training
Minhyun Lee (Yonsei University), Hyunjung Shim
ClassificationSegmentationComputational EfficiencyTransformerAuto EncoderImage
🎯 What it does: Building upon SeI_T, this paper proposes a self-supervised pre-training scheme called Masked Token Modeling (MTM), combined with two novel token-level augmentation methods, TokenAdapt and ColorAdapt, to construct a complete storage-efficient visual training framework (SeiT++).
Select and Distill: Selective Dual-Teacher Knowledge Transfer for Continual Learning on Vision-Language Models
Yu-Chu Yu (National Taiwan University), Yu-Chiang Frank Wang (National Taiwan University)
Knowledge DistillationVision Language ModelContrastive LearningImage
🎯 What it does: Proposes the Selective Dual-Teacher Knowledge Transfer framework, achieving simultaneous avoidance of catastrophic forgetting and maintenance of zero-shot inference capability in continual learning for vision-language models.
SelEx: Self-Expertise in Fine-Grained Generalized Category Discovery
Sarah Rastegar (University of Amsterdam), Cees Snoek
ClassificationTransformerContrastive LearningImage
🎯 What it does: Propose the SelEx method, combining unsupervised and supervised self-expertise techniques, generating multi-level pseudo labels through hierarchical semi-supervised k-means to achieve a fine-grained general category discovery (GCD) framework for simultaneously discovering and classifying known and unknown categories.
Self-Adapting Large Visual-Language Models to Edge Devices across Visual Modalities
Kaiwen Cai (University College London), Chris Xiaoxuan Lu (Cisco Research)
Computational EfficiencyKnowledge DistillationRepresentation LearningTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Propose the EdgeVL framework, which can migrate large-scale vision-language (VL) models (e.g., CLIP) to edge devices without manual annotation, and is compatible with RGB and non-RGB (e.g., depth, infrared) multimodal inputs, supporting open-vocabulary classification.