ICCV 2023 Papers — Page 18
IEEE/CVF International Conference on Computer Vision · 2156 papers
Sentence Attention Blocks for Answer Grounding
Seyedalireza Khoshsirat (University of Delaware), Chandra Kambhamettu (University of Delaware)
RecognitionObject DetectionTransformerVision Language ModelImageText
🎯 What it does: A Sentence Attention Block is proposed, which achieves precise localization of answer-related visual areas by recalibrating the image feature channels in the answer localization task.
Sequential Texts Driven Cohesive Motions Synthesis with Natural Transitions
Shuai Li (Beihang University), Aimin Hao (Beihang University)
GenerationData SynthesisPose EstimationAuto EncoderVideoTextSequential
🎯 What it does: A framework for human motion sequence synthesis based on free-form sequential text is proposed, ensuring semantic connections and natural transitions between adjacent actions.
Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models
Dong Lu (Southern University of Science and Technology), Feng Zheng (Monash University)
RetrievalAdversarial AttackTransformerVision Language ModelMultimodality
🎯 What it does: This study investigates the adversarial transferability of visual-language pre-trained models and proposes the Set-level Guidance Attack (SGA) method.
SFHarmony: Source Free Domain Adaptation for Distributed Neuroimaging Analysis
Nicola K Dinsdale (University of Oxford), Ana IL Namburete (University of Oxford)
ClassificationSegmentationDomain AdaptationImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A source-free domain adaptation method called SFHarmony is proposed, which can achieve domain unification and feature alignment for MRI data without accessing the source data.
SG-Former: Self-guided Transformer with Evolving Token Reallocation
Sucheng Ren (National University of Singapore), Xinchao Wang (National University of Singapore)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: A self-guided Transformer (SG-Former) is designed to dynamically reallocate tokens based on the importance map predicted by the model itself, achieving efficient fine-grained global self-attention modeling through a mixed scale attention mechanism that integrates both local and global attention within the same layer.
SGAligner: 3D Scene Alignment with Scene Graphs
Sayan Deb Sarkar (ETH Zurich), Iro Armeni (ETH Zurich)
Object DetectionGraph Neural NetworkContrastive LearningPoint CloudGraphTime Series
🎯 What it does: This paper proposes a framework called SGAligner based on 3D scene graph alignment, which can align nodes of two 3D scene graphs under unknown overlap or environmental changes, and apply this alignment result to point cloud registration, stitching, and map alignment.
SHACIRA: Scalable HAsh-grid Compression for Implicit Neural Representations
Sharath Girish (University of Maryland), Kamal Gupta (University of Maryland)
CompressionImageVideo
🎯 What it does: This paper presents SHACIRA, an end-to-end trainable framework for compressing implicit neural representations (INR) based on multi-resolution feature grids.
Shape Analysis of Euclidean Curves under Frenet-Serret Framework
Perrine Chassat (University of Paris-Saclay), Nicolas Brunel (Quantmetry)
Point CloudTime Series
🎯 What it does: This paper proposes a Euclidean curve shape analysis method based on the Frenet-Serret framework and introduces a new Square-Root Curvatures (SRC) transformation to fully capture the geometric features of curves.
Shape Anchor Guided Holistic Indoor Scene Understanding
Mingyue Dong (Wuhan University), Xianwei Zheng (Wuhan University)
Object DetectionSegmentationPoint CloudMesh
🎯 What it does: This paper proposes a shape anchor point guided learning strategy called AncLearn, and integrates it into the AncRec framework to achieve unified indoor scene understanding from detection to reconstruction.
ShapeScaffolder: Structure-Aware 3D Shape Generation from Text
Xi Tian (University of Bath), Qi Wu (University of Adelaide)
GenerationData SynthesisGraph Neural NetworkTransformerTextPoint CloudMesh
🎯 What it does: Designed ShapeScaffolder, a structure-aware text-to-3D shape generation model that achieves fine-grained correspondence by utilizing the internal hierarchical structure of shapes and text dependency graphs;
Shatter and Gather: Learning Referring Image Segmentation with Text Supervision
Dongwon Kim (POSTECH), Suha Kwak (POSTECH)
RecognitionSegmentationTransformerContrastive LearningImageText
🎯 What it does: A weakly supervised referential image segmentation method is proposed, which utilizes only image-text pairs. The model discovers visual entities through bottom-up attention and generates segmentation masks by combining text through top-down attention.
SHERF: Generalizable Human NeRF from a Single Image
Shoukang Hu (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
GenerationData SynthesisPose EstimationTransformerNeural Radiance FieldImage
🎯 What it does: We propose SHERF, a generalizable Human NeRF that can recover animatable 3D human bodies from a single portrait image.
Shift from Texture-bias to Shape-bias: Edge Deformation-based Augmentation for Robust Object Recognition
Xilin He (Shenzhen University), Linlin Shen (Shenzhen University)
RecognitionObject DetectionData SynthesisConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: An online data augmentation method based on edge deformation (SDbOA) is proposed, which diversifies the shape of objects through TPS deformation, thereby reducing the texture bias of CNNs and enhancing the model's reliance on shape features.
SHIFT3D: Synthesizing Hard Inputs For Tricking 3D Detectors
Hongge Chen (Cruise LLC), Yuning Chai (Cruise LLC)
Object DetectionAutonomous DrivingAdversarial AttackPoint Cloud
🎯 What it does: We propose SHIFT3D, a differentiable pipeline that generates natural and challenging 3D objects by performing gradient optimization on 3D shapes and poses in the DeepSDF latent space, and inserts them into LiDAR point cloud scenes.
ShiftNAS: Improving One-shot NAS via Probability Shift
Mingyang Zhang (Zhejiang University of Technology), Linlin Ou (Zhejiang University of Technology)
OptimizationNeural Architecture SearchConvolutional Neural NetworkTransformerImage
🎯 What it does: Proposes ShiftNAS, a One-shot NAS method that dynamically adjusts the sampling probability of sub-networks through probability shift, and designs a differentiable LSTM architecture generator;
Shortcut-V2V: Compression Framework for Video-to-Video Translation Based on Temporal Redundancy Reduction
Chaeyeon Chung (KAIST), Jaegul Choo (KAIST)
Data SynthesisCompressionKnowledge DistillationConvolutional Neural NetworkGenerative Adversarial NetworkVideo
🎯 What it does: A general compression framework called Shortcut-V2V is proposed, which can accelerate video-to-video translation models without significant loss of quality.
Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning
Lihe Yang (University of Hong Kong), Hengshuang Zhao (University of Hong Kong)
ClassificationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes a semi-supervised learning framework called ShrinkMatch, which reduces the category space by automatically removing categories that are confused with the highest predicted category, allowing uncertain samples to gain sufficient confidence in the new space and be utilized;
SIDGAN: High-Resolution Dubbed Video Generation via Shift-Invariant Learning
Urwa Muaz (Amazon Prime Video), Naveen Nair (Amazon Prime Video)
GenerationData SynthesisGenerative Adversarial NetworkVideo
🎯 What it does: Achieved high-resolution video lip synchronization generation while maintaining identity and pose.
SIGMA: Scale-Invariant Global Sparse Shape Matching
Maolin Gao (Technical University of Munich), Florian Bernard (University of Bonn)
OptimizationMesh
🎯 What it does: A sparse non-rigid shape matching framework called SIGMA based on mixed-integer programming is proposed, which can achieve sparse correspondence while ensuring global optimality and is invariant to scale and rigid transformations.
Sigmoid Loss for Language Image Pre-Training
Xiaohua Zhai (Google DeepMind), Lucas Beyer (Google DeepMind)
Representation LearningVision Language ModelContrastive LearningImageText
🎯 What it does: A simple pairwise sigmoid loss is proposed for image-text pre-training, aimed at improving the efficiency and effectiveness of contrastive learning.
Sign Language Translation with Iterative Prototype
Huijie Yao (University of Science and Technology of China), Houqiang Li (University of Science and Technology of China)
Image TranslationKnowledge DistillationTransformerVideoText
🎯 What it does: The IP-SLT framework is proposed, which utilizes an iterative prototype refinement mechanism to continuously aggregate the visual features of sign language videos with the previous round of prototypes, ultimately achieving a more accurate semantic representation and completing text translation. At the same time, an iterative distillation loss is introduced during training, allowing intermediate iterations to gain supervision from the final iteration, thereby improving translation quality.
SiLK: Simple Learned Keypoints
Pierre Gleize (Meta AI), Matt Feiszli (Meta AI)
RecognitionObject DetectionRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: A simple and scalable self-supervised learning framework SiLK has been designed and implemented for learning image keypoint detection and description.
SILT: Shadow-Aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels
Han Yang (Chinese University of Hong Kong), Chi-Wing Fu (Chinese University of Hong Kong)
Object DetectionSegmentationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A Shadow-aware Iterative Label Tuning (SILT) framework is proposed, which iteratively refines noisy shadow labels through a self-training approach, significantly improving shadow detection accuracy.
SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning
Hao Feng (University of Science and Technology of China), Houqiang Li (University of Science and Technology of China)
RestorationRepresentation LearningTransformerContrastive LearningOptical FlowImage
🎯 What it does: A simple framework called SimFIR based on self-supervised representation learning is proposed for the de-distortion processing of fisheye images.
Similarity Min-Max: Zero-Shot Day-Night Domain Adaptation
Rundong Luo (Peking University), Jiaying Liu (Peking University)
ClassificationSegmentationDomain AdaptationContrastive LearningImage
🎯 What it does: A zero-shot day-night domain adaptation similarity minimax framework is proposed, which first generates nighttime images with the lowest similarity to daytime image features using an exposure-guided image translation module, and then maximizes the feature similarity between the two domains through contrastive learning, thereby enhancing the model's robustness at night.
SimMatchV2: Semi-Supervised Learning with Graph Consistency
Mingkai Zheng (University of Sydney), Chang Xu (University of Sydney)
ClassificationRepresentation LearningGraph Neural NetworkContrastive LearningImage
🎯 What it does: A semi-supervised learning framework named SimMatchV2 is proposed, modeling different augmented views as graph nodes and achieving joint learning of labeled and unlabeled samples through four types of consistency.
SimNP: Learning Self-Similarity Priors Between Neural Points
Christopher Wewer (Max Planck Institute for Informatics), Jan Eric Lenssen (Max Planck Institute for Informatics)
RestorationGenerationNeural Radiance FieldAuto EncoderPoint Cloud
🎯 What it does: This paper proposes a renderable light field based on neural points (SimNP), achieving high-quality 3D reconstruction under single-view and dual-view conditions through self-attention learning of category-level self-similarity priors.
Simoun: Synergizing Interactive Motion-appearance Understanding for Vision-based Reinforcement Learning
Yangru Huang (Peking University), Yonghong Tian
Autonomous DrivingConvolutional Neural NetworkReinforcement LearningContrastive LearningVideo
🎯 What it does: This paper presents Simoun, a dual-path network framework that separately learns motion and appearance features from visual observations, achieving more efficient visual reinforcement learning through a structural interaction module and a consistency-driven curiosity module.
Simple and Effective Out-of-Distribution Detection via Cosine-based Softmax Loss
SoonCheol Noh (Sungkyunkwan University), Jee-Hyong Lee (Sungkyunkwan University)
ClassificationAnomaly DetectionConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A deep classification model trained based on cosine softmax loss is proposed, utilizing a combination of feature norm and Mahalanobis distance to construct an OOD detection method that requires no hyperparameters, no additional data, and no input processing.
Simple Baselines for Interactive Video Retrieval with Questions and Answers
Kaiqu Liang (Princeton University), Samuel Albanie (University of Cambridge)
RetrievalTransformerLarge Language ModelVision Language ModelVideoText
🎯 What it does: This paper proposes an interactive video retrieval framework based on question answering, utilizing a video question answering model to simulate user responses for interactive retrieval.
SimpleClick: Interactive Image Segmentation with Simple Vision Transformers
Qin Liu (University of North Carolina at Chapel Hill), Marc Niethammer (University of North Carolina at Chapel Hill)
SegmentationTransformerImageBiomedical Data
🎯 What it does: An interactive image segmentation method called SimpleClick is proposed, which is based on a standard ViT backbone and achieves efficient segmentation through symmetric embedding of click information and a simple feature pyramid.
Simulating Fluids in Real-World Still Images
Siming Fan (SenseTime Research), Kwan-Yee Lin (Shanghai AI Laboratory)
GenerationData SynthesisConvolutional Neural NetworkOptical FlowImageVideo
🎯 What it does: This paper proposes a method for generating realistic fluid animations from a single static image. The core idea is to decompose the scene into a surface fluid layer and an opaque background layer, and based on this, combine 2.5D fluid simulation with deep learning motion prediction to achieve fluid animation and interactive editing.
SINC: Self-Supervised In-Context Learning for Vision-Language Tasks
Yi-Syuan Chen (National Yang Ming Chiao Tung University), Hong-Han Shuai (National Yang Ming Chiao Tung University)
Representation LearningMeta LearningTransformerVision Language ModelImageTextMultimodality
🎯 What it does: A framework named SINC is proposed, which trains a meta-model on frozen multi-source features (visual, linguistic, cross-modal) using self-supervised constructed examples (LID, DID, OD) to achieve immediate context learning for visual-language tasks.
SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation
Nikos Athanasiou (Max Planck Institute for Intelligent Systems), Gül Varol (LIGM, École des Ponts, Université Gustave Eiffel)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVideoText
🎯 What it does: This paper proposes a 3D human motion generation method called SINC (Spatial Composition of 3D Human Motions) that executes multiple actions based on text descriptions. It achieves multi-action spatial composition by automatically constructing training data for combinable actions.
Single Depth-image 3D Reflection Symmetry and Shape Prediction
Zhaoxuan Zhang, Xin Yang
RestorationReinforcement LearningPoint Cloud
🎯 What it does: Iterative shape completion using reflective symmetry on a single depth map, first estimating the symmetry plane and symmetrically aligning the point cloud, then selecting virtual reflective viewpoints through reinforcement learning to generate depth/normal maps, using 2D inpainting (including normal assistance) to fill in the missing parts, and finally projecting back to 3D and updating the point cloud, iterating multiple times until convergence.
Single Image Deblurring with Row-dependent Blur Magnitude
Xiang Ji (University of Tokyo), Yinqiang Zheng (National Institute of Informatics)
RestorationTransformerImageVideo
🎯 What it does: For the single-frame image deblurring task under the GRR (Global Reset Release) shutter mode, a Transformer-based RSS-T block is proposed, which can utilize row-related blur features to recover clear images.
Single Image Defocus Deblurring via Implicit Neural Inverse Kernels
Yuhui Quan (South China University of Technology), Hui Ji (National University of Singapore)
RestorationRecurrent Neural NetworkImage
🎯 What it does: An end-to-end interpretable single image defocus deblurring network called INIKNet is proposed, which directly predicts the inverse convolution kernel of spatial variations and performs deblurring.
Single Image Reflection Separation via Component Synergy
Qiming Hu (Tianjin University), Xiaojie Guo (Tianjin University)
RestorationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a new single-image reflection separation model that utilizes a learnable residual term and a dual-stream semantic-aware network to achieve more complete separation of reflection and transmission layers.
Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction
Hansheng Chen (Tongji University), Hao Su (University of California)
GenerationData SynthesisDiffusion modelNeural Radiance FieldAuto EncoderPoint Cloud
🎯 What it does: Proposes SSDNeRF, a unified framework for single-stage training that can simultaneously perform unconditional 3D generation and sparse view reconstruction;
SIRA-PCR: Sim-to-Real Adaptation for 3D Point Cloud Registration
Suyi Chen (University of Electronic Science and Technology of China), Shuaicheng Liu (University of Electronic Science and Technology of China)
RecognitionData SynthesisDomain AdaptationConvolutional Neural NetworkGenerative Adversarial NetworkPoint Cloud
🎯 What it does: A simulation-to-real adaptation framework SIRA-PCR is proposed for 3D point cloud registration.
Size Does Matter: Size-aware Virtual Try-on via Clothing-oriented Transformation Try-on Network
Chieh-Yun Chen (Stylins.ai), Wen-Huang Cheng (National Taiwan University)
Image TranslationSegmentationGenerationConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A high-resolution virtual try-on network based on garment structure, COTTON, is proposed, which can achieve precise deformation through garment key points and segmentation, and supports try-on for different sizes.
SKED: Sketch-guided Text-based 3D Editing
Aryan Mikaeili (Simon Fraser University), Ali Mahdavi-Amiri (Simon Fraser University)
GenerationOptimizationDiffusion modelNeural Radiance FieldImagePoint Cloud
🎯 What it does: Based on existing NeRF models, local 3D editing is achieved using a small number of multi-view hand-drawn sketches and text prompts.
SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training
Hong Yan (Sun Yat-sen University), Liang Lin (Sun Yat-sen University)
RecognitionGraph Neural NetworkAuto EncoderGraphSequential
🎯 What it does: This paper proposes SkeletonMAE, a graph-based masked autoencoder for pre-training skeleton sequences, and combines it with the STRL module to construct an SSL framework for action recognition.
SkeleTR: Towards Skeleton-based Action Recognition in the Wild
Haodong Duan (Chinese University of Hong Kong), Alessandro Bergamo (AWS AI Labs)
RecognitionPose EstimationGraph Neural NetworkTransformerVideo
🎯 What it does: This paper proposes the SkeleTR framework, a two-stage model (GCN + Transformer) based on short skeleton sequences for multi-person skeleton action recognition.
Sketch and Text Guided Diffusion Model for Colored Point Cloud Generation
Zijie Wu (Hunan University), Ajmal Mian (University of Western Australia)
GenerationData SynthesisDiffusion modelPoint Cloud
🎯 What it does: This paper proposes a probabilistic diffusion model based on sketches and text conditions for generating colored point clouds, achieving joint generation from hand-drawn sketches and natural language descriptions to 3D shapes and their appearances.
Skill Transformer: A Monolithic Policy for Mobile Manipulation
Xiaoyu Huang, Andrew Szot
Robotic IntelligenceTransformerReinforcement LearningImage
🎯 What it does: This paper proposes Skill Transformer, an end-to-end transformer strategy that decomposes long-term mobile operation tasks into skill prediction and low-level control.
Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning
Zhiheng Li (Tsinghua University), Jie Zhou (Tsinghua University)
TransformerVideo
🎯 What it does: Proposes Skip-Plan, which learns by skipping intermediate state supervision and condensing the action space, breaking long action chains into several reliable sub-chains, and predicting and aggregating them into a complete action sequence using sub-chain decoders.
SKiT: a Fast Key Information Video Transformer for Online Surgical Phase Recognition
Yang Liu (King's College London), Sebastien Ourselin (King's College London)
RecognitionComputational EfficiencyTransformerVideo
🎯 What it does: A fast key information Transformer named SKiT is proposed for online surgical phase recognition.
SlaBins: Fisheye Depth Estimation using Slanted Bins on Road Environments
Jongsung Lee (UNIST), Kyungdon Joo (UNIST)
Depth EstimationAutonomous DrivingTransformerImage
🎯 What it does: This paper proposes a monocular fisheye camera depth estimation network called SlaBins based on Slanted Multi-Cylinder Images (MCI).
SLAN: Self-Locator Aided Network for Vision-Language Understanding
Jiang-Tian Zhai (Nankai University), Ming-Ming Cheng (Nankai University)
Object DetectionRetrievalTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This work proposes the Self-Locator Aided Network (SLAN), which achieves text-driven region extraction and fine-grained visual-language alignment through a self-locator without the need for gold annotations.
SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model
Gengwei Zhang (University of Technology Sydney), Yunchao Wei (Institute of Information Science Beijing Jiaotong University)
ClassificationRepresentation LearningTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes a method called Slow Learner with Classifier Alignment (SLCA) for continual learning on pre-trained models.
Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning
Xiang Yuan (Northwestern Polytechnical University), Junwei Han (Northwestern Polytechnical University)
Object DetectionConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes CFINet, a two-stage detection framework specifically designed for small objects;
SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-Training
Yuanze Lin (University of Washington), Cihang Xie (University of California Santa Cruz)
RetrievalComputational EfficiencyTransformerVision Language ModelAuto EncoderVideoTextMultimodality
🎯 What it does: In the video-language pre-training task, SMAUG proposes an efficient framework that utilizes Masked Autoencoder (MAE); it jointly masks visual and textual data based on MAE and further reduces computational load through Visual Token Sparsification and Frame Selector.
SMMix: Self-Motivated Image Mixing for Vision Transformers
Mengzhao Chen (Xiamen University), Rongrong Ji (Xiamen University)
Object DetectionSegmentationTransformerImage
🎯 What it does: A self-driven image mixing method called SMMix is proposed to enhance the generalization and robustness of Vision Transformers.
Smoothness Similarity Regularization for Few-Shot GAN Adaptation
Vadim Sushko (Bosch Center for Artificial Intelligence), Juergen Gall (University of Bonn)
GenerationDomain AdaptationGenerative Adversarial NetworkImage
🎯 What it does: This study focuses on few-shot GAN adaptation, proposing smooth similarity regularization and improved discriminator loss to achieve high-quality and diverse image generation from pre-trained GANs to the target domain.
Snow Removal in Video: A New Dataset and A Novel Method
Haoyu Chen (Hong Kong University of Science and Technology), Lei Zhu (Hong Kong University of Science and Technology)
RestorationData SynthesisContrastive LearningOptical FlowVideo
🎯 What it does: This paper addresses the video de-snowing task, first constructing the first high-quality video de-snowing dataset (RVSD), and then proposing a deep learning-based de-snowing framework.
SOAR: Scene-debiasing Open-set Action Recognition
Yuanhao Zhai (University at Buffalo), Gang Hua (Wormpex AI Research)
RecognitionAdversarial AttackConvolutional Neural NetworkVideo
🎯 What it does: This paper proposes a scene debiasing method for open set action recognition (SOAR), which suppresses background information through adversarial scene reconstruction and adaptive scene classification of video features, allowing the model to focus more on action foregrounds, thereby improving the recognition of unknown actions and the determination of known actions.
Social Diffusion: Long-term Multiple Human Motion Anticipation
Julian Tanke (University of Bonn), Cem Keskin (Reality Labs Research)
GenerationPose EstimationDiffusion modelVideo
🎯 What it does: A multi-person human action prediction framework based on diffusion models—Social Diffusion—has been proposed, which generates motion sequences feasible for social interaction while maintaining the authenticity of individual postures.
SOCS: Semantically-Aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variations
Boyan Wan (National University of Defense Technology), Kai Xu (National University of Defense Technology)
Pose EstimationGraph Neural NetworkContrastive LearningPoint Cloud
🎯 What it does: A semantic alignment object coordinate space (SOCS) is proposed for category-level 6D pose and size estimation;
SoDaCam: Software-defined Cameras via Single-Photon Imaging
Varun Sundar (University of Wisconsin Madison), Mohit Gupta (University of Wisconsin Madison)
RestorationCompressionImageVideo
🎯 What it does: Using a single-photon avalanche diode (SPAD) array to capture a photon cube, a series of simple linear and shift projection operations are proposed to generate various imaging modes from the same data in post-processing: high frame rate video compressed imaging, event camera, motion projection (equivalent to camera movement during exposure), etc.
Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation
Ziyang Chen (University of Michigan), Andrew Owens (University of Michigan)
Representation LearningConvolutional Neural NetworkContrastive LearningMultimodalityAudio
🎯 What it does: A self-supervised joint learning method for sound source localization and camera rotation (SLfM) is proposed, achieved through the geometric consistency of visual and binaural audio generated by head rotation;
Sound Source Localization is All about Cross-Modal Alignment
Arda Senocak (Korea Advanced Institute of Science and Technology), Joon Son Chung (Korea Advanced Institute of Science and Technology)
RecognitionRetrievalContrastive LearningMultimodalityAudio
🎯 What it does: This study investigates the importance of cross-modal semantic understanding in sound source localization and proposes a self-supervised method that combines cross-modal alignment with multi-view positive sample construction.
Source-free Depth for Object Pop-out
Zongwei WU, Luc Van Gool (ETH Zurich)
Object DetectionSegmentationDomain AdaptationConvolutional Neural NetworkImage
🎯 What it does: Learn contact surfaces through source-free deep networks and the 'pop-out' prior, converting depth to semantics to achieve cross-domain and cross-task object segmentation.
Source-free Domain Adaptive Human Pose Estimation
Qucheng Peng (University of Central Florida), Chen Chen (University of Central Florida)
Pose EstimationDomain AdaptationContrastive LearningImage
🎯 What it does: This paper proposes a source-free domain adaptive human pose estimation task and designs an end-to-end framework consisting of a source model, a transition model, and a target model in a three-layer structure;
Space Engage: Collaborative Space Supervision for Contrastive-Based Semi-Supervised Semantic Segmentation
Changqi Wang (Northeastern University), Xiangyu Yue (Chinese University of Hong Kong)
SegmentationContrastive LearningImage
🎯 What it does: A Cooperative Space Supervision (CSS) framework is proposed, which utilizes logit space and representation space to jointly generate pseudo-labels and perform semi-supervised semantic segmentation on unlabeled images.
Space-time Prompting for Video Class-incremental Learning
Yixuan Pei (Xi'an Jiaotong University), Xueming Qian (Xi'an Jiaotong University)
ClassificationRecognitionTransformerPrompt EngineeringVision Language ModelVideo
🎯 What it does: In the task of video category incremental learning, the CLIP pre-trained model is utilized to achieve replay-free incremental learning by learning various spatial-temporal prompts (task-specific prompts and task-agnostic prompts).
SPACE: Speech-driven Portrait Animation with Controllable Expression
Siddharth Gururani (NVIDIA), Ming-Yu Liu (NVIDIA)
GenerationData SynthesisPose EstimationRecurrent Neural NetworkAuto EncoderVideoAudio
🎯 What it does: Generate high-resolution, controllable expression videos of single portraits using voice, supporting adjustments for head pose, blinking, gaze, and emotional intensity.
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
Xudong Wang (Shanghai Jiao Tong University), Mao Yang (Microsoft)
Computational EfficiencyKnowledge DistillationNeural Architecture SearchImage
🎯 What it does: Proposes the SpaceEvo method, which automatically constructs a hardware-specific INT8 quantization-friendly search space and trains a quantization-for-all super network within that space, ultimately resulting in the SEQnet series of efficient models.
Spacetime Surface Regularization for Neural Dynamic Scene Reconstruction
Jaesung Choe (NVIDIA), Anima Anandkumar (California Institute of Technology)
GenerationData SynthesisNeural Radiance FieldVideo
🎯 What it does: A 4D dynamic scene reconstruction method based on spatiotemporal surface regularization, 4DRegSDF, is proposed, utilizing a deformable Signed Distance Function (SDF) combined with local rigidity constraints to achieve high-fidelity neural rendering and geometric reconstruction.
SPANet: Frequency-balancing Token Mixer using Spectral Pooling Aggregation Modulation
Guhnoo Yun (Korea Institute of Science and Technology), Dong Hwan Kim (Korea Institute of Science and Technology)
ClassificationObject DetectionSegmentationTransformerContrastive LearningImage
🎯 What it does: This paper designs a Token Mixer (SPAM) that achieves high-low frequency balance in the frequency domain, and based on this, constructs the SPANet network, aiming to enhance the visual model's ability to capture information at different frequencies.
Sparse Instance Conditioned Multimodal Trajectory Prediction
Yonghao Dong (Xi'an Jiaotong University), Gang Hua (Wormpex AI Research)
Object TrackingAutonomous DrivingRecurrent Neural NetworkMultimodalityTime Series
🎯 What it does: A Sparse Instance Conditional Network (SICNet) is proposed for multimodal pedestrian trajectory prediction, utilizing sparse instances (representative points of future trajectories) to guide the predictions.
Sparse Point Guided 3D Lane Detection
Chengtang Yao (Beijing Institute of Technology), Yunde Jia (Shenzhen MSU-BIT University)
Object DetectionAutonomous DrivingTransformerPoint Cloud
🎯 What it does: A sparse point-guided 3D lane detection framework is proposed, which achieves high-precision lane positioning through coarse detection at low resolution followed by refinement using sparse points.
Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks
Sixiang Chen (Hong Kong University of Science and Technology), Lei Zhu (Hong Kong University of Science and Technology)
RestorationTransformerImage
🎯 What it does: This paper proposes a sparse sampling Transformer (UDR-S Former) to achieve unified rain removal for raindrops and rain streaks, utilizing uncertainty ranking to guide attention sampling, thereby adaptively modeling the relationship of rainfall damage on a global scale.
SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videos
Haisong Liu (Nanjing University), Limin Wang (Nanjing University)
Object DetectionAutonomous DrivingTransformerVideo
🎯 What it does: This paper proposes a completely sparse multi-camera video 3D object detection framework called SparseBEV, which performs detection directly in the BEV space without the need to construct dense BEV features.
SparseDet: Improving Sparsely Annotated Object Detection with Pseudo-positive Mining
Saksham Suri (University of Maryland), Abhinav Shrivastava (University of Maryland)
Object DetectionImage
🎯 What it does: For the task of object detection with sparse annotations, an end-to-end SparseDet framework is proposed to enhance detection performance through pseudo-positive sample mining and self-supervised consistency learning.
SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detection
Yichen Xie (University of California), Wei Zhan (University of California)
Object DetectionAutonomous DrivingTransformerImageMultimodalityPoint Cloud
🎯 What it does: Using sparse candidate boxes and sparse representation, we construct a 3D object detection framework called SparseFusion for LiDAR and camera sensors. Instance-level features are extracted through parallel single-modal detectors, and then the camera candidate boxes are projected into the LiDAR coordinate system. A lightweight self-attention module is used to fuse the two modal features in a unified 3D space, ultimately yielding high-quality 3D frames.
SparseMAE: Sparse Training Meets Masked Autoencoders
Aojun Zhou (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)
ClassificationObject DetectionSegmentationTransformerAuto EncoderImage
🎯 What it does: Proposes the SparseMAE method, which achieves efficient pre-training on a small Vision Transformer by combining sparse training with Masked Autoencoder.
SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis
Guangcong Wang, Ziwei Liu
Data SynthesisDepth EstimationNeural Radiance FieldImage
🎯 What it does: Using a pre-trained single-view depth model or rough depth maps from consumer-grade sensors, SparseNeRF is proposed for novel view synthesis with a limited number of views.
Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxes
Di Wu (University of Chinese Academy of Sciences), Jianbin Jiao (University of Chinese Academy of Sciences)
Object DetectionKnowledge DistillationImage
🎯 What it does: To address inaccurate bounding box annotations, an end-to-end self-distillation detection framework called SSD-Det is proposed to enhance object detection performance in the absence of high-quality annotations.
Spatial-Aware Token for Weakly Supervised Object Localization
Pingyu Wu (University of Science and Technology of China), Zheng-Jun Zha (University of Science and Technology of China)
Object DetectionSegmentationTransformerImage
🎯 What it does: This paper proposes a weakly supervised object localization method based on Transformer, utilizing Spatial-Aware Tokens (SAT) to directly generate localization maps through spatial query attention, avoiding optimization conflicts between classification and localization tasks.
Spatially and Spectrally Consistent Deep Functional Maps
Mingze Sun (Tsinghua University), Ruqi Huang (Tsinghua University)
Diffusion modelPoint CloudMesh
🎯 What it does: Proposes a two-branch unsupervised deep feature mapping (DFM) framework that integrates spectral domain and spatial domain cyclic consistency to enhance non-rigid shape correspondence.
Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution
Long Sun (Nanjing University of Science and Technology), Jinshan Pan (Nanjing University of Science and Technology)
RestorationSuper ResolutionConvolutional Neural NetworkImage
🎯 What it does: A lightweight single-image super-resolution network named SAFMN is proposed for efficient super-resolution on low-power devices.
Spatio-Temporal Crop Aggregation for Video Representation Learning
Sepehr Sameni (University of Bern), Paolo Favaro (University of Bern)
RetrievalComputational EfficiencyRepresentation LearningTransformerContrastive LearningVideoAgriculture Related
🎯 What it does: A video clipping aggregation framework SCALE based on a pre-trained backbone is proposed, utilizing sparse clipping, mask clipping modeling, and global CLS contrastive learning to achieve efficient video representation learning while maintaining low computational costs.
Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception
Kun Yang (Fudan University), Liang Song (Fudan University)
Object DetectionAutonomous DrivingRecurrent Neural NetworkSimultaneous Localization and MappingPoint Cloud
🎯 What it does: This paper proposes an end-to-end multi-agent collaborative perception framework named SCOPE, aimed at enhancing the 3D object detection performance of autonomous vehicles.
Spatio-temporal Prompting Network for Robust Video Feature Extraction
Guanxiong Sun (Queen's University Belfast), Yang Hua (Queen's University Belfast)
Object DetectionObject TrackingSegmentationTransformerPrompt EngineeringVideo
🎯 What it does: A unified Spatio-Temporal Prompt Network (STPN) is proposed, achieving robust video feature extraction by injecting dynamic video prompts at the front end of the Transformer, eliminating complex backend integration modules.
Spectral Graphormer: Spectral Graph-Based Transformer for Egocentric Two-Hand Reconstruction using Multi-View Color Images
Tze Ho Elden Tse (Google), Bardia Doosti (Google)
SegmentationPose EstimationOptimizationGraph Neural NetworkTransformerImageMesh
🎯 What it does: A Transformer framework based on spectral graph convolution is proposed to reconstruct high-quality hand meshes (including forearms) from multi-view RGB images and eliminate self-intersections through optimization.
Spectrum-guided Multi-granularity Referring Video Object Segmentation
Bo Miao (University of Western Australia), Ajmal Mian (University of Western Australia)
Object DetectionSegmentationTransformerVideo
🎯 What it does: A spectrum-guided multi-granularity R-VOS framework SgMg is proposed to address the feature drift problem and achieve single-frame/multi-frame/multi-object segmentation.
Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Video
Xiuzhe Wu (University of Hong Kong), Xiaojuan Qi (University of Hong Kong)
GenerationData SynthesisContrastive LearningVideo
🎯 What it does: A Speech2Lip framework is proposed, which generates high-fidelity, synchronized speaker videos from just a few minutes of video by separating speech-sensitive and speech-insensitive motions, utilizing implicit models, geometric mapping, hybrid networks, and contrastive synchronization loss.
Speech4Mesh: Speech-Assisted Monocular 3D Facial Reconstruction for Speech-Driven 3D Facial Animation
Shan He (University of Science and Technology of China), Chang Xu (University of Sydney)
GenerationData SynthesisTransformerContrastive LearningVideoMeshAudio
🎯 What it does: Proposes the Speech4Mesh framework, which generates pseudo 4D talking head data using voice information from monocular video, and subsequently trains an audio-driven 3D facial animation model;
Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution
Zixiang Zhao (Xi'an Jiaotong University), Luc Van Gool (ETH Zurich)
RestorationDepth EstimationSuper ResolutionTransformerContrastive LearningImage
🎯 What it does: A new SSDNet based on spherical spatial feature decomposition and contrastive learning is proposed to guide the super-resolution reconstruction of depth images.
SpinCam: High-Speed Imaging via a Rotating Point-Spread Function
Dorian Chan (Carnegie Mellon University), Matthew O'Toole (Carnegie Mellon University)
RestorationCompressionOptical FlowImageVideo
🎯 What it does: By installing a rotatable diffraction grating in front of the camera, the point spread function (PSF) of the camera changes over time. Using a single frame image captured with a global shutter, deconvolution reconstruction is performed to obtain high frame rate video.
SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes
Yutao Cui (Nanjing University), Limin Wang (Nanjing University)
Object DetectionObject TrackingTransformerVideoBenchmark
🎯 What it does: A large-scale multi-object tracking dataset, SportsMOT, covering basketball, volleyball, and soccer scenarios has been constructed, and the MixSort framework has been proposed to enhance tracking performance.
Spurious Features Everywhere - Large-Scale Detection of Harmful Spurious Features in ImageNet
Yannic Neuhaus (Tübingen AI Center University of Tübingen), Matthias Hein (Tübingen AI Center University of Tübingen)
ClassificationObject DetectionAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This paper systematically discovers and verifies harmful interference features on ImageNet using class-level neural PCA methods and visualization techniques, and proposes the SpuFix correction scheme, which reduces the dependency of any ImageNet classifier on these interference features without the need for additional annotations or retraining.
SQAD: Automatic Smartphone Camera Quality Assessment and Benchmarking
Zilin Fang (National University of Singapore), Radu Timofte (ETH Zurich)
ClassificationConvolutional Neural NetworkTransformerImageBenchmark
🎯 What it does: The SQAD dataset is proposed, which conducts six quality assessments (resolution, color accuracy, noise, dynamic range, PSF, and aliasing) based on laboratory measurements for 29 smartphone cameras, and trains deep models to achieve automatic camera quality assessment and device identification.
SRFormer: Permuted Self-Attention for Single Image Super-Resolution
Yupeng Zhou (Nankai University), Qibin Hou (Nankai University)
RestorationSuper ResolutionTransformerImage
🎯 What it does: A Permuted Self-Attention (PSA) mechanism is proposed, and based on this, a single-image super-resolution network called SRFormer is constructed.
SSB: Simple but Strong Baseline for Boosting Performance of Open-Set Semi-Supervised Learning
Yue Fan (Max Planck Institute for Informatics), Bernt Schiele (Max Planck Institute for Informatics)
ClassificationObject DetectionSupervised Fine-TuningImage
🎯 What it does: A simple yet powerful baseline (SSB) is proposed in open-set semi-supervised learning, achieving improvements in both in-class classification and outlier detection through three main strategies: high-confidence pseudo-labeling, nonlinear feature projection heads, and pseudo-negative sample mining.
SSDA: Secure Source-Free Domain Adaptation
Sabbir Ahmed (Binghamton University), Adnan Siraj Rakin (North Carolina State University)
Domain AdaptationAdversarial AttackImage
🎯 What it does: This study investigates the risks of backdoor attacks in source-free domain adaptation (SFDA) and proposes a secure training scheme called SSDA.
SSF: Accelerating Training of Spiking Neural Networks with Stabilized Spiking Flow
Jingtao Wang (University of Chinese Academy of Sciences), Zhaoxiang Zhang (Tencent)
Spiking Neural NetworkImage
🎯 What it does: This paper proposes a method called Stabilized Spiking Flow (SSF), which utilizes a stabilization approximation of the spike flow in SNNs to skip the temporal backpropagation of membrane potential, significantly accelerating the training of SNNs based on Surrogate Gradient.
Stabilizing Visual Reinforcement Learning via Asymmetric Interactive Cooperation
Yunpeng Zhai (Peking University), Yonghong Tian (Peking University)
Autonomous DrivingConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: A framework for Asynchronous Interactive Cooperation (AIC) is proposed, which interacts lightweight encoders with reparameterized re-encoders to address the instability in training large models in visual reinforcement learning.
Stable and Causal Inference for Discriminative Self-supervised Deep Visual Representations
Yuewei Yang (Duke University), Yiran Chen (Duke University)
Representation LearningContrastive LearningImage
🎯 What it does: This paper explores the impact of causal instability on downstream tasks in discriminative self-supervised learning and proposes two correction methods during the inference phase to enhance robustness.