NeurIPS 2024 Papers — Page 34
Conference on Neural Information Processing Systems · 4035 papers
SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal Fusion
Ming Dai (Southeast University), Wankou Yang (Southeast University)
RecognitionObject DetectionKnowledge DistillationTransformerVision Language ModelImageMultimodality
🎯 What it does: A concise and efficient visual grounding framework named SimVG is proposed, which decouples multimodal fusion from downstream tasks and directly uses a pretrained multimodal model for feature interaction.
Single Image Reflection Separation via Dual-Stream Interactive Transformers
Qiming Hu (Tianjin University), Xiaojie Guo (Tianjin University)
RestorationConvolutional Neural NetworkTransformerImage
🎯 What it does: A dual-stream interactive Transformer for single image reflection separation is proposed, utilizing global and local prior interactions and introducing a dual attention module to achieve inter-layer and intra-layer feature collaboration.
Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models
Jiaqi Li (Southeast University), Fan Liu (Hohai University)
RecognitionData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningImageMultimodalityBenchmark
🎯 What it does: Proposes a Single Image Unlearning (SIU) method that utilizes a single target concept image and multi-faceted fine-tuning data to achieve visual recognition unlearning in multimodal large language models.
Single-Loop Stochastic Algorithms for Difference of Max-Structured Weakly Convex Functions
Quanqi Hu (Texas A&M University), Tianbao Yang (Texas A&M University)
OptimizationImage
🎯 What it does: A unified DMax optimization framework is proposed, covering differential weakly convex (DWC) optimization and weakly convex-strongly concave (WCSC) min-max optimization, and a single-loop stochastic Moreau envelope approximate gradient method (SMAG) is designed to implement this framework.
SIRIUS : Contexual Sparisty with Correction for Efficient LLMs
Yang Zhou (Carnegie Mellon University), Beidi Chen (Carnegie Mellon University)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper studies the use of Contextual Sparsity technology in large language model inference to reduce computational load and proposes the SIRIUS mechanism to correct erroneous tokens generated during the sparse model generation process, thereby restoring inference quality.
Sketched Lanczos uncertainty score: a low-memory summary of the Fisher information
Marco Miani (Technical University of Denmark), Søren Hauberg (Technical University of Denmark)
ClassificationOptimizationComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: A low-memory Fisher information matrix approximation method is proposed for pre-trained neural networks, and it is used to calculate uncertainty scores (SLU).
Sketching for Distributed Deep Learning: A Sharper Analysis
Mayank Shrivastava (University of Illinois Urbana-Champaign), Arindam Banerjee (University of Illinois Urbana-Champaign)
OptimizationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: This paper presents a convergence analysis for the use of random sketch compression in distributed deep learning, leveraging the second-order properties of deep models to eliminate dependence on model dimensions.
Sketchy Moment Matching: Toward Fast and Provable Data Selection for Finetuning
Yijun Dong (New York University), Qi Lei (New York University)
OptimizationData-Centric LearningSupervised Fine-TuningImage
🎯 What it does: A two-stage SkMM method is proposed for data selection in fine-tuning tasks, which explores the parameter space through gradient sketching and then performs moment matching to balance variance and bias.
SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions
Zizhao Wang (University of Texas at Austin), Peter Stone (University of Texas at Austin)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: An unsupervised skill learning framework called SkiLD based on local dependencies is proposed, which can efficiently learn diverse interactive skills in a factored state space and be used for subsequent sparse reward tasks.
Skill-aware Mutual Information Optimisation for Zero-shot Generalisation in Reinforcement Learning
Xuehui Yu (Harbin Institute of Technology), Stefano V Albrecht
Robotic IntelligenceMeta LearningReinforcement LearningContrastive Learning
🎯 What it does: This paper studies a context encoder based on skill-aware mutual information, aimed at enhancing the zero-shot generalization ability of Meta-RL across different tasks.
Skinned Motion Retargeting with Dense Geometric Interaction Perception
Zijie Ye (Tsinghua University), Mike Zheng Shou (National University of Singapore)
TransformerMesh
🎯 What it does: Proposes the MeshRet framework, achieving skinning motion redirection based on dense geometric interaction perception.
SkipPredict: When to Invest in Predictions for Scheduling
Rana Shahout (Harvard University), Michael Mitzenmacher (Harvard University)
OptimizationTabularTime Series
🎯 What it does: Designed and analyzed the SkipPredict scheduling strategy, optimizing average response time in an M/G/1 queue system by considering prediction costs through dual-level prediction (cheap and expensive).
Slack-Free Spiking Neural Network Formulation for Hypergraph Minimum Vertex Cover
Tam Ngoc-Bang Nguyen (Australian Institute for Machine Learning), Tat-Jun Chin (Australian Institute for Machine Learning)
OptimizationSpiking Neural NetworkGraph
🎯 What it does: A spiking neural network (SNN) without slack variables is proposed to solve the hypergraph minimum vertex cover (HMVC) problem.
SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models
Jianyi Zhang (Duke University), Yiran Chen (Duke University)
GenerationOptimizationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a novel decoding framework—Self Logits Evolution Decoding (SLED), which enhances the factual accuracy of model outputs by self-evolving the logits of LLM during the inference phase.
SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents
Ethan Rathbun (Northeastern University), Alina Oprea (Northeastern University)
Adversarial AttackReinforcement LearningMultimodality
🎯 What it does: A new dynamic reward backdoor attack method called SleeperNets is proposed, targeting backdoor induction during the reinforcement learning (RL) training process, which can force the agent to execute specified actions upon triggering while maintaining good performance.
Slicing Vision Transformer for Flexible Inference
Yitian Zhang (Snap Inc.), Yun Fu (Northeastern University)
ClassificationSegmentationKnowledge DistillationTransformerImageVideo
🎯 What it does: Proposes the Scala framework, allowing a single Vision Transformer to be sliced by width for flexible inference;
Slight Corruption in Pre-training Data Makes Better Diffusion Models
Hao Chen (Carnegie Mellon University), Bhiksha Raj (William and Mary)
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: The study investigates the impact of mild contamination of conditions (such as labels and text) during the pre-training phase of diffusion models and proposes enhancing model performance by adding perturbations to the condition embeddings (CEP).
SLIM: Style-Linguistics Mismatch Model for Generalized Audio Deepfake Detection
Yi Zhu (Reality Defender Inc), Gaurav Bharaj (Reality Defender Inc)
Anomaly DetectionExplainability and InterpretabilityContrastive LearningAudio
🎯 What it does: Proposes the SLIM (Style-Linguistics Mismatch) model, which detects audio deepfakes through a two-stage process;
SlimGPT: Layer-wise Structured Pruning for Large Language Models
Gui Ling (Alibaba Group), Qingwen Liu (Alibaba Group)
CompressionOptimizationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a structured pruning method called SlimGPT, which implements hierarchical pruning of LLMs using the Optimal Brain Surgeon framework, and introduces batch greedy pruning and incremental pruning ratios to enhance pruning effectiveness.
SlimSAM: 0.1% Data Makes Segment Anything Slim
Zigeng Chen (National University of Singapore), Xinchao Wang (National University of Singapore)
SegmentationCompressionKnowledge DistillationImage
🎯 What it does: This paper proposes SlimSAM, a method for compressing the Segment Anything Model (SAM) using very little training data (0.1%).
Slot State Space Models
Jindong Jiang (Rutgers University), Sungjin Ahn (KAIST)
TransformerVideoBenchmark
🎯 What it does: This paper introduces Slot State Space Models (SlotSSMs), which incorporate modular slot states into the traditional SSM framework to achieve more efficient and parallel modeling of long sequences.
Slot-VLM: Object-Event Slots for Video-Language Modeling
Jiaqi Xu (University of Science and Technology of China), Yan Lu (Microsoft Research Asia)
RecognitionObject DetectionSegmentationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoText
🎯 What it does: Designed and implemented the Slot-VLM framework, which utilizes a dual-branch Slot Attention (object slots and event slots) to generate a small number of semantically decoupled visual tokens from video features for video reasoning by large language models (LLMs).
SLowcalSGD : Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
Tehila Dahan (Technion), Kfir Yehuda Levy
OptimizationTabular
🎯 What it does: A new local update algorithm SLo wcal-SGD is proposed for heterogeneous distributed stochastic convex optimization, significantly improving communication efficiency and convergence speed in multi-machine training.
SlowFocus: Enhancing Fine-grained Temporal Understanding in Video LLM
Ming Nie (Fudan University), Li Zhang (Fudan University)
TransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoBenchmark
🎯 What it does: The SlowFocus mechanism is proposed, which enhances the understanding and reasoning of fine-grained temporal information in video LLMs through high-frequency sampling, temporal encoding, and multi-frequency mixed attention during query-related periods.
SLTrain: a sparse plus low rank approach for parameter and memory efficient pretraining
Andi Han (RIKEN Artificial Intelligence Project), Bamdev Mishra (Microsoft)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: The SLTrain method is proposed, which decomposes the weight matrix into low-rank and fixed random sparse parts during the pre-training phase of large language models, achieving high efficiency in both parameters and memory.
Sm: enhanced localization in Multiple Instance Learning for medical imaging classification
Francisco M Castro-Macías, Aggelos Katsaggelos
ClassificationObject DetectionTransformerImageBiomedical DataComputed Tomography
🎯 What it does: A new smoothing operator Sm is proposed to enhance the localization capability of multi-instance learning (MIL) in medical image classification, particularly in instance-level predictions.
Small coresets via negative dependence: DPPs, linear statistics, and concentration
Rémi Bardenet (University of Lille), Hoang-Son Tran
Image
🎯 What it does: This paper constructs a small coreset using negatively correlated Deterministic Point Processes (DPP) and proves that it outperforms independent sampling in terms of uniformly approximating the error of the original loss in the parameter space.
Small steps no more: Global convergence of stochastic gradient bandits for arbitrary learning rates
Jincheng Mei (Google DeepMind), Dale Schuurmans (Google DeepMind)
OptimizationReinforcement LearningTabular
🎯 What it does: Proves that the stochastic gradient game algorithm almost surely converges to the global optimal strategy under any constant learning rate.
SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
Yu Yang (University of California), Baharan Mirzasoleiman (University of California)
TransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Proposed the SMALLTOLARGE (S2L) method: train a small model to obtain the loss trajectory for each sample, cluster the trajectories, and then uniformly sample from each cluster to obtain a subset for supervised fine-tuning of a large model.
SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction
Wei Wu (Tsinghua University), Yuheng KAN
GenerationAutonomous DrivingTransformerLarge Language ModelSequential
🎯 What it does: A self-regressive motion generation framework called SMART based on discrete sequences is proposed, which uses a GPT-style decoder-only transformer to directly predict the next action or road vector label.
SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction
Zhihao Yu (Peking University), Junfeng Zhao (Peking University)
TransformerSupervised Fine-TuningTabularBiomedical DataElectronic Health Records
🎯 What it does: The SMART model is proposed, utilizing self-supervised missingness-aware pre-training to enhance predictive performance on EHR data.
Smoke and Mirrors in Causal Downstream Tasks
Riccardo Cadei (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
TransformerVideoBenchmark
🎯 What it does: This study investigates how training design and prediction processes when using pre-trained deep learning models for causal downstream tasks may lead to biased causal effect estimates, validated based on a newly constructed high-dimensional observational dataset called ISTAnt.
Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention
Susung Hong (University of Washington)
GenerationData SynthesisTransformerDiffusion modelGaussian SplattingImage
🎯 What it does: A training-free, unconditional Smoothed Energy Guidance (SEG) method is proposed, which utilizes Gaussian blur on self-attention weights to reduce energy curvature, thereby enhancing the image generation quality of diffusion models.
Smoothed Online Classification can be Harder than Batch Classification
Vinod Raman (University of Michigan), Ambuj Tewari (University of Michigan)
ClassificationOptimization
🎯 What it does: In the smoothed online classification environment, the distinction between PAC learnability and smoothed learnability is explored when the label space is infinite. A hypothesis class is constructed that is PAC learnable but not learnable in the smoothed environment; a sufficient condition for smoothed learning, UBEME, is provided, proving its generality in relation to PAC learnability. The necessity and sufficiency of UBEME are further discussed.
Smoothie: Label Free Language Model Routing
Neel Guha (Stanford University), Christopher Re
GenerationTransformerLarge Language ModelText
🎯 What it does: A label-free routing algorithm called SMOOTHIE is proposed, which automatically estimates the quality of each LLM on each input using similarity information output by LLMs and routes to the best model.
SnapKV: LLM Knows What You are Looking for Before Generation
Yuhong Li (University of Illinois Urbana-Champaign), Deming Chen (University of Illinois Urbana-Champaign)
GenerationCompressionComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes SnapKV, a KV cache compression method that does not require fine-tuning, which can identify and retain the most important attention features in advance during the generation phase, significantly reducing time and memory consumption during long text inference.
SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization
Wanhua Li (Harvard University), Hanspeter Pfister (Harvard University)
OptimizationExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: Proposes the SocialGPT framework, which utilizes visual foundation models to extract image information, generates symbolic social stories, and then uses large language models for social relationship reasoning to provide interpretable answers.
SocraticLM: Exploring Socratic Personalized Teaching with Large Language Models
Jiayu Liu (University of Science and Technology of China), Enhong Chen (University of Science and Technology of China)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Proposes SocraticLM, which implements Socratic thinking-based personalized teaching by combining the SocraTeach dataset.
Soft ascent-descent as a stable and flexible alternative to flooding
Matthew J. Holland (Osaka University), Kosuke Nakatani (Osaka University)
ClassificationOptimizationImage
🎯 What it does: A soft adaptive gradient method called SoftAD is designed and evaluated to improve the generalization and model complexity of traditional Flooding and SAM in classification tasks.
Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space
Leo Schwinn (Technical University of Munich), Stephan Günnemann (Technical University of Munich)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes and evaluates embedding space adversarial attacks on open-source large language models, demonstrating their ability to efficiently bypass safety alignment, recover 'forgotten' information, and extract pre-training data.
Soft Superpixel Neighborhood Attention
Kent Gauen (Purdue University), Stanley H. Chan
RestorationImage
🎯 What it does: A soft superpixel neighborhood attention (SNA) module is proposed for image denoising tasks, which re-weights attention weights using pixel-level superpixel probabilities to better capture the variable boundaries of objects.
Soft-Label Integration for Robust Toxicity Classification
Zelei Cheng (Northwestern University), Xinyu Xing (Northwestern University)
ClassificationOptimizationTransformerLarge Language ModelText
🎯 What it does: A dual-layer optimization-based soft label ensemble framework is proposed, utilizing crowdsourced annotations to enhance the robustness of toxic text classification.
SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion
Lu Han (Nanjing University), De-Chuan Zhan (Nanjing University)
Computational EfficiencyRecurrent Neural NetworkGraph Neural NetworkTransformerReinforcement LearningTime Series
🎯 What it does: A multivariate time series forecasting model SOFTS based on MLP is proposed, which introduces the STAR module to capture inter-channel correlations for efficient prediction.
SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model
Grzegorz Stefański (Samsung AI Center), Jakub Tkaczuk (Samsung AI Center)
RecognitionComputational EfficiencyConvolutional Neural NetworkAudio
🎯 What it does: Proposes the Scattered Online Inference (SOI) method, which utilizes compression and prediction of partial network states to reduce the computational complexity of real-time convolutional neural networks.
Solving Inverse Problems via Diffusion Optimal Control
Henry Li (Yale University), Marcus Aloysius Pereira
RestorationSuper ResolutionOptimizationDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A framework for solving inverse problems based on discrete optimal control (Diffusion Optimal Control) is constructed, treating the reverse diffusion process as controllable dynamics and directly searching for samples in the control space that satisfy observation constraints.
Solving Minimum-Cost Reach Avoid using Reinforcement Learning
Oswin So (Massachusetts Institute of Technology), Chuchu Fan (Massachusetts Institute of Technology)
OptimizationReinforcement Learning
🎯 What it does: Proposes the RC-PPO algorithm to solve the minimum cost reach-avoid problem using reinforcement learning.
Solving Sparse \& High-Dimensional-Output Regression via Compression
Renyuan Li (National University of Singapore), Guanyi Wang (National University of Singapore)
OptimizationComputational EfficiencyGaussian SplattingTabular
🎯 What it does: A two-stage compression framework for the multi-output regression (SHORE) problem with high-dimensional sparse outputs is proposed, which first compresses the outputs through random projection and then trains and performs projection gradient descent for prediction.
Solving Zero-Sum Markov Games with Continuous State via Spectral Dynamic Embedding
Chenhao Zhou (Zhejiang University), Hui Qian (Zhejiang University)
Reinforcement Learning
🎯 What it does: A natural policy gradient algorithm named SDEPO is proposed to solve two-player zero-sum Markov games with continuous state spaces and finite action spaces, along with its theoretical convergence guarantees.
SongCreator: Lyrics-based Universal Song Generation
Shun Lei (Tsinghua University), Helen M. Meng
GenerationTransformerDiffusion modelTextAudio
🎯 What it does: Designed and implemented the SongCreator system, which can automatically generate complete songs based on lyrics, including vocals and accompaniment, and supports various generation and editing tasks.
Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases
Zian Su (Purdue University), Xiangyu Zhang (Purdue University)
GenerationAI Code AssistantTransformerLarge Language ModelMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes a 'probe-and-recover' framework called ProRec, which combines a binary-source code cross-modal alignment encoder-decoder (Prober) with a black-box LLM (Recoverer) to automatically generate symbol-rich source code snippets as context, enhancing the effectiveness of binary reverse engineering tasks (summary and function name recovery).
Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation
Julius Vetter (University of Tübingen), Jakob H. Macke (Max Planck Institute for Intelligent Systems)
Biomedical Data
🎯 What it does: This paper proposes a source distribution estimation method (Sourcerer) based on the maximum entropy principle, which infers the parameter distribution of scientific simulators from observational data without requiring explicit likelihood.
Space-Time Continuous PDE Forecasting using Equivariant Neural Fields
David M Knigge, Stratis Gavves
Computational EfficiencyMeta LearningGraph Neural NetworkTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: This paper proposes a spatiotemporal continuous PDE solving framework based on Equivariant Neural Fields (ENF), which learns equivariant ODE flows in the latent space, maintaining the known symmetries and boundary conditions of the PDE while supporting sparse and irregular sampling of initial conditions.
SpaceByte: Towards Deleting Tokenization from Large Language Modeling
Kevin Slagle (Rice University)
TransformerLarge Language ModelText
🎯 What it does: A byte-level Transformer decoder architecture named SpaceByte is proposed to eliminate the tokenization step in large language models.
SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead
Minsu Kim (Virginia Tech), Choong Seon Hong (Kyung Hee University)
Federated LearningComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: The SpaFL framework is proposed, which achieves structured sparsity through trainable thresholds, only communicating thresholds in federated learning rather than parameters, significantly reducing communication and computation costs while improving model accuracy.
Span-Based Optimal Sample Complexity for Weakly Communicating and General Average Reward MDPs
Matthew Zurek (University of Wisconsin Madison), Yudong Chen (University of Wisconsin Madison)
OptimizationReinforcement Learning
🎯 What it does: This paper studies the sample complexity of learning ε-optimal policies in average reward Markov Decision Processes (MDPs) under generative models, and provides optimal upper bounds for weakly communicating and general (multi-chain) MDPs;
SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization
Shuchen Zhu (Peking University), Kun Yuan (Peking University)
OptimizationMeta LearningReinforcement LearningImage
🎯 What it does: This paper presents SPARKLE, a unified single-loop primal-dual framework for solving decentralized bilevel optimization problems.
Sparse Bayesian Generative Modeling for Compressive Sensing
Benedikt Böck (Technical University of Munich), Wolfgang Utschick (Technical University of Munich)
GenerationCompressionAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a Sparse Bayesian Generative Modeling approach for linear inverse problems in compressed sensing, which can learn from a small number of noisy compressed samples and directly solve inverse problems without optimization.
Sparse High Rank Adapters
Kartikeya Bhardwaj (Qualcomm AI Research), Markus Nagel (Qualcomm AI Research)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelImageText
🎯 What it does: A highly sparse high-rank adapter (SHiRA) is proposed, which can complete various generation tasks by fine-tuning only 1-2% of the parameters of the pre-trained model.
Sparse maximal update parameterization: A holistic approach to sparse training dynamics
Nolan Simran Dey (Cerebras Systems), Joel Hestness (Cerebras Systems)
TransformerLarge Language ModelText
🎯 What it does: Proposed Sparse Maximum Update Parameterization (SµPar) to stabilize the training dynamics of sparse networks and achieve performance surpassing that of dense networks.
Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis
Qitao Zhao (Carnegie Mellon University), Shubham Tulsiani (Carnegie Mellon University)
Pose EstimationDiffusion modelPoint Cloud
🎯 What it does: Jointly inferring camera pose and 3D structure in sparse views, achieving high-quality reconstruction using an analysis-synthesis framework combined with generative priors.
SparseLLM: Towards Global Pruning of Pre-trained Language Models
Guangji Bai (Emory University), Liang Zhao (Argonne National Laboratory)
OptimizationTransformerLarge Language ModelText
🎯 What it does: Proposes the SparseLLM framework, which breaks down the global pruning of large-scale LLMs into manageable subproblems, achieving globally optimal pruning under low resources;
Sparsity-Agnostic Linear Bandits with Adaptive Adversaries
Tianyuan Jin (National University of Singapore), Nicolò Cesa-Bianchi (Università degli Studi di Milano)
OptimizationReinforcement LearningTabular
🎯 What it does: This study investigates the algorithms and theories of sparse oblivious linear Bandits under adaptive adversaries generating action sets.
SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors
Chenyang Ma (University of Oxford), Andrew Markham (University of Oxford)
Depth EstimationRobotic IntelligenceTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: The SpatialPIN framework is designed to enhance the spatial reasoning capabilities of VLM through prompts and interaction with various 3D foundational models without training, and it is applied to spatial VQA and robotic tasks.
SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models
An-Chieh Cheng (University of California San Diego), Sifei Liu (NVIDIA)
RecognitionObject DetectionDepth EstimationTransformerLarge Language ModelVision Language ModelImageMultimodalityBenchmark
🎯 What it does: Proposes the SpatialRGPT model to enhance the capabilities of VLM in regional spatial reasoning and 3D perception.
Spatio-Spectral Graph Neural Networks
Simon Geisler (Munich Data Science Institute), Stephan Günnemann (Munich Data Science Institute)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes a new graph neural network architecture S² GNN, which combines spatial information propagation (MPGNN) and frequency domain spectral filtering to achieve stronger long-range interaction modeling and expressive capability.
Spatio-Temporal Interactive Learning for Efficient Image Reconstruction of Spiking Cameras
Bin Fan (Peking University), Boxin Shi (Peking University)
RestorationSpiking Neural NetworkOptical FlowImageVideo
🎯 What it does: An efficient spatiotemporal interactive network (STIR) is proposed for reconstructing high-quality intermediate frames from the binary spatiotemporal pulse stream of an integrated neuromorphic vision camera.
Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication
Olaf Lipinski (University of Southampton), Timothy J. Norman (University of Southampton)
RecognitionExplainability and InterpretabilityRecurrent Neural NetworkAgentic AISequential
🎯 What it does: In a reference game environment, agents communicate spatial relationships of target positions in a sequence through discrete messages and analyze their interpretability.
SpeAr: A Spectral Approach for Zero-Shot Node Classification
Ting Guo (North University of China), Jianchao Zeng (North University of China)
ClassificationRepresentation LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: Proposes the SpeAr method, which utilizes spectral analysis and learnable class prototypes to achieve zero-shot node classification, employing a two-stage training process and introducing spectral contrastive loss to mine the clustering structure of unlabeled nodes.
SPEAR: Exact Gradient Inversion of Batches in Federated Learning
Dimitar Iliev Dimitrov, Martin Vechev (ETH Zurich)
Federated LearningComputational EfficiencyAdversarial AttackImage
🎯 What it does: An algorithm named SPEAR is proposed, which can accurately reconstruct input data with a batch size greater than 1 in federated learning, challenging the previous assumption that this is not achievable in an honest-but-curious setting.
Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting
Ziyi Yang (Zhejiang University), Xiaogang Jin (Zhejiang University)
GenerationData SynthesisComputational EfficiencyNeural Radiance FieldGaussian SplattingPoint Cloud
🎯 What it does: The Spec-Gaussian method is proposed, which changes the perspective-related appearance modeling of 3D Gaussian splatting from low-order spherical harmonics to anisotropic spherical Gaussians (ASG) and combines it with a coarse-to-fine training strategy, significantly improving the rendering effects for specular highlights and anisotropic materials.
SpecExec: Massively Parallel Speculative Decoding For Interactive LLM Inference on Consumer Devices
Ruslan Svirschevski (Yandex), Max Ryabinin (Together AI)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes a large-scale language model inference method for consumer GPUs called SpecExec, which accelerates interactive inference through parallel speculative decoding and parameter offloading.
Spectral Adapter: Fine-Tuning in Spectral Space
Fangzhao Zhang (Stanford University), Mert Pilanci (Stanford University)
TransformerSupervised Fine-TuningDiffusion modelText
🎯 What it does: Proposes the Spectral Adapter, which achieves low-parameter fine-tuning by performing incremental or orthogonal rotations in the singular vector space of the pre-trained weights;
Spectral Editing of Activations for Large Language Model Alignment
Yifu QIU, Shay B Cohen
GenerationOptimizationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a training-agnostic activation editing method—Spectral Editing of Activations (SEA), which guides the model to generate outputs that better align with human preferences (more accurate and fair) by projecting the internal representations of LLM during inference onto directions that are maximally correlated with positive examples (e.g., true) and minimally correlated with negative examples (e.g., false).
Spectral Graph Pruning Against Over-Squashing and Over-Smoothing
Adarsh Jamadandi (Universität des Saarlandes), Rebekka Burkholz (CISPA Helmholtz Center for Information Security)
OptimizationGraph Neural NetworkGraph
🎯 What it does: This paper proposes using spectral graph pruning (edge deletions) to simultaneously alleviate over-squashing and over-smoothing in graph neural networks, and designs a greedy pruning framework based on proxy spectral gap updates, PROXYDELETE / PROXYADD, for rapid evaluation and pruning.
Spectral Learning of Shared Dynamics Between Generalized-Linear Processes
Lucine L Oganesian, Maryam Shanechi
OptimizationTime Series
🎯 What it does: This paper proposes a multi-stage covariance-based system identification algorithm PGLDM, which is used to simultaneously model two generalized linear time series and explicitly separate shared and private dynamics.
Spectral-Risk Safe Reinforcement Learning with Convergence Guarantees
Dohyeong Kim (Seoul National University), Songhwai Oh (Seoul National University)
OptimizationSafty and PrivacyReinforcement LearningTabular
🎯 What it does: A safety reinforcement learning algorithm based on spectral risk measures is proposed—Spectral-Risk-Constrained Policy Optimization (SRCPO), which ensures convergence and optimality in discrete (tabular) environments through bi-level optimization and achieves optimal performance in continuous control tasks.
Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
Zhuofan Wen (Institute of Computing Technology, Chinese Academy of Sciences), Yang Feng (Institute of Computing Technology, Chinese Academy of Sciences)
GenerationComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: A CTC-based drafting model (CTC-drafter) is proposed for speculative decoding to accelerate LLM inference.
Speculative Monte-Carlo Tree Search
Scott Cheng (Pennsylvania State University), Ding-Yong Hong (Academia Sinica)
Reinforcement LearningTabular
🎯 What it does: Proposed and evaluated Speculative MCTS, a technique for achieving inter-decision parallelism during the self-play phase of AlphaZero;
SpeechAlign: Aligning Speech Generation to Human Preferences
Dong Zhang (Fudan University), Xipeng Qiu (Fudan University)
GenerationOptimizationReinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement LearningAudio
🎯 What it does: Construct and iteratively optimize a speech generation model to make its output more aligned with human preferences for speech quality, naturalness, and timbre.
SpeechForensics: Audio-Visual Speech Representation Learning for Face Forgery Detection
Yachao Liang (Institute of Information Engineering Chinese Academy of Sciences), Weiqing Huang (Institute of Information Engineering Chinese Academy of Sciences)
Anomaly DetectionRepresentation LearningContrastive LearningVideoMultimodalityAudio
🎯 What it does: Proposes an unsupervised facial forgery detection method that identifies forgeries by detecting semantic mismatches between lip movements in the video and audio through audiovisual speech representation learning from real videos.
SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation
Yiqi Zhang (National University of Singapore), Yang You (National University of Singapore)
TransformerLarge Language ModelText
🎯 What it does: We propose SpeedLoader, an I/O efficient solution for training and inference of heterogeneous and distributed large language models, which significantly reduces model state communication through hierarchical scheduling and activation caching.
SpelsNet: Surface Primitive Elements Segmentation by B-Rep Graph Structure Supervision
Kseniya Cherenkova (University of Luxembourg), Djamila Aouada (University of Luxembourg)
ClassificationSegmentationConvolutional Neural NetworkGraph Neural NetworkPoint Cloud
🎯 What it does: We propose SpelsNet, an end-to-end network that can directly predict the types and topological structures of faces, edges, and vertices in Boundary Representation (B-Rep) from 3D point clouds.
SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard Attentive Spiking Neural Network
Weiyu Guo (Hong Kong University of Science and Technology), Hui Xiong (Hong Kong University of Science and Technology)
RecognitionDomain AdaptationSpiking Neural NetworkTime Series
🎯 What it does: A SpGesture framework based on spiking neural networks is proposed, which includes Spiking Jaccard Attention and Source-Free Domain Adaptation, specifically addressing the distribution shift problem in sEMG gesture recognition.
Spherical Frustum Sparse Convolution Network for LiDAR Point Cloud Semantic Segmentation
Yu Zheng (Shanghai Jiao Tong University), Hesheng Wang (Shanghai Jiao Tong University)
SegmentationAutonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: A LiDAR point cloud semantic segmentation method based on Spherical Frustum, SFCNet, is proposed, which eliminates the loss of geometric information caused by quantization in traditional spherical projection.
Spike-based Neuromorphic Model for Sound Source Localization
Dehao Zhang (University of Electronic Science and Technology of China), Yang Yang (University of Electronic Science and Technology of China)
RecognitionSpiking Neural NetworkAudio
🎯 What it does: A sound source localization framework based on pulse neural networks is proposed, utilizing RF-PLC to achieve energy-efficient ITD encoding, and enhancing robustness through the MAA module (frequency-space joint attention and short-term memory).
SpikedAttention: Training-Free and Fully Spike-Driven Transformer-to-SNN Conversion with Winner-Oriented Spike Shift for Softmax Operation
Sangwoo Hwang (Daegu Gyeongbuk Institute of Science and Technology), Jaeha Kung (Korea University)
Spiking Neural NetworkTransformerImageText
🎯 What it does: A training-free, fully synaptic Transformer to SNN conversion method called SpikedAttention is proposed.
SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams
Kang Chen (Peking University), Zhaofei Yu (Peking University)
RestorationSuper ResolutionKnowledge DistillationSpiking Neural NetworkImageVideo
🎯 What it does: A self-supervised spike-guided motion deblurring framework S-SDM has been developed, which can utilize low-resolution spike flow to recover a continuous sequence of clear frames from a single blurred image.
Spiking Graph Neural Network on Riemannian Manifolds
Li Sun (North China Electric Power University), Philip S. Yu (University of Illinois at Chicago)
OptimizationComputational EfficiencyGraph Neural NetworkSpiking Neural NetworkGraphOrdinary Differential Equation
🎯 What it does: A spike graph neural network (MSG) that operates in Riemannian geometric spaces is proposed, achieving energy-efficient learning of graph data.
Spiking Neural Network as Adaptive Event Stream Slicer
Jiahang Cao (Hong Kong University of Science and Technology), Renjing Xu (Hong Kong University of Science and Technology)
Object TrackingSpiking Neural NetworkVideoPoint Cloud
🎯 What it does: An adaptive event stream segmentation method called SpikeSlicer is proposed, utilizing low-energy pulse neural networks (SNN) as event triggers.
Spiking Token Mixer: An event-driven friendly Former structure for spiking neural networks
Shikuang Deng (University of Electronic Science and Technology of China), Shi Gu (University of Electronic Science and Technology of China)
ClassificationOptimizationComputational EfficiencyNeural Architecture SearchSpiking Neural NetworkTransformerImage
🎯 What it does: A novel event-driven friendly spiking neural network architecture called STMixer is proposed, which achieves high-performance inference at T=1 by implementing token mixing and information-preserving patch segmentation using operations such as convolution, fully connected layers, and residual paths that are only supported on asynchronous hardware.
Spiking Transformer with Experts Mixture
Zhaokun Zhou (Peking University), Li Yuan (Peking University)
Spiking Neural NetworkTransformerMixture of ExpertsImage
🎯 What it does: A specialist mixture mechanism (SEMM) that can be directly implemented in spiking neural networks (SNN) has been designed and embedded into a spiking Transformer, forming the EMSA and EMSP modules.
Splatter a Video: Video Gaussian Representation for Versatile Processing
Yang-Tian Sun (University of Hong Kong), XIAOJUAN QI
Object TrackingDepth EstimationKnowledge DistillationRepresentation LearningGaussian SplattingOptical FlowVideo
🎯 What it does: This paper proposes an explicit Video Gaussian Representation (VGR), which embeds videos into three-dimensional Gaussians and assigns time-related three-dimensional motion attributes to each Gaussian, supporting various processing tasks such as video reconstruction, dense tracking, depth/feature consistency, geometric/appearance editing, frame interpolation, viewpoint synthesis, and stereo video generation.
SplitNeRF: Split Sum Approximation Neural Field for Joint Geometry, Illumination, and Material Estimation
Jesus Zarzar (King Abdullah University of Science and Technology), Bernard Ghanem (King Abdullah University of Science and Technology)
RestorationGenerationOptimizationComputational EfficiencyNeural Radiance FieldImage
🎯 What it does: A NeRF-based inverse rendering framework is proposed, which utilizes split sum approximation to separate lighting and material, and learns pre-integrated lighting and occlusion factors through a single MLP, allowing for the simultaneous estimation of object geometry, material properties, and environmental lighting within a few hours.
SPO: Sequential Monte Carlo Policy Optimisation
Matthew Macfarlane, Alexandre Laterre (InstaDeep)
OptimizationReinforcement LearningSequential
🎯 What it does: This paper proposes Sequential Monte Carlo Policy Optimisation (SPO), which achieves adaptive improvement of policies by combining SMC planning with the EM framework.
SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learning
Huy Hoang (Singapore Management University), Pradeep Varakantham (Singapore Management University)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: A new offline imitation learning algorithm called SPRINQL is proposed, which can simultaneously utilize expert and multi-level suboptimal demonstrations for learning.
SS1: Accelerating Inference with Fast and Expressive Sketch Structured Transform
Aditya Desai (Rice University), Anshumali Shrivastava (Rice University)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A new structured linear transformation method SS1 is proposed, which accelerates the inference of the linear layers in deep learning models using random parameter sharing and GPU-friendly computation.
SSA-Seg: Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentation
Xiaowen Ma (Huawei Noah's Ark Lab), Xinghao Chen (Huawei Noah's Ark Lab)
SegmentationKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a pixel-level classifier called SSA-Seg, which is based on semantic and spatial adaptation. It uses rough masks to guide the original prototypes to shift towards the semantic and spatial centers of the test images, and enhances performance through online multi-domain distillation.
SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening
Yu Zhong (University of Electronic Science and Technology of China), Hong-Xia Dou (Xihua University)
Image TranslationRestorationDiffusion modelImage
🎯 What it does: This paper proposes a diffusion model called SSDiff based on spatial-spectral subspace decomposition, which uses a dual-branch network to learn spatial details and spectral features separately, and achieves high-quality remote sensing image fusion through an alternating projection fusion module.
SSDM: Scalable Speech Dysfluency Modeling
Jiachen Lian (University of California Berkeley), Gopala Anumanchipalli (University of California Berkeley)
RecognitionTransformerLarge Language ModelAudio
🎯 What it does: The SSDM framework is proposed, which achieves scalable speech fluency modeling and detection through a neural motion posture model (gestural VAE), connected subsequence alignment (CSA), and a large language model.
ST$_k$: A Scalable Module for Solving Top-k Problems
Hanchen Xia (Shanghai Jiao Tong University), Xiaojun Mao (Shanghai Jiao Tong University)
ClassificationOptimizationTransformerSupervised Fine-TuningImageTextTabular
🎯 What it does: This paper proposes a differentiable Smoothed Top-k (STk) module that can achieve Top-k ranking in O(n+k) time within neural networks and embeds it into the loss function, smoothing the Average Top-k (AT-k) loss to obtain the STk loss.