ICML 2025 Papers — Page 27
International Conference on Machine Learning · 3257 papers
Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning
Yuhui Wang (King Abdullah University of Science and Technology), Jürgen Schmidhuber (Swiss AI Lab IDSIA/USI/SUPSI)
Autonomous DrivingOptimizationConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: This paper proposes a Dynamic Transfer VIN (DT-VIN), which introduces a state and observation-based dynamic transfer kernel and an adaptive highway loss, allowing the value iteration network to be trained up to 5000 layers, thus solving extremely long-term planning tasks.
Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillation
Chuanqi Cheng (Renmin University of China), Rui Yan (Renmin University of China)
RetrievalCompressionComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelVision Language ModelVideoMultimodality
🎯 What it does: We propose VILAMP, a hierarchical video-language model that can process up to 10K frames (approximately 2.7 hours) of long videos on a single NVIDIA A100 GPU through differential keyframe selection and differential feature merging.
SCENIR: Visual Semantic Clarity through Unsupervised Scene Graph Retrieval
Nikolaos Chaidos (National Technical University of Athens), Giorgos Stamou (National Technical University of Athens)
RetrievalGraph Neural NetworkAuto EncoderGraph
🎯 What it does: This paper proposes an unsupervised scene graph retrieval framework (SCENIR) based on graph autoencoders, using Graph Edit Distance (GED) as a deterministic criterion for assessing retrieval similarity.
SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data via Scalable Conditioned Neural Fields
David Keetae Park (Brookhaven National Laboratory), Shinjae Yoo (Brookhaven National Laboratory)
TransformerTime SeriesPhysics Related
🎯 What it does: A unified SCENT framework is proposed, capable of simultaneously achieving reconstruction of sparse inputs, spatial interpolation, and temporal prediction.
Schwarz–Schur Involution: Lightspeed Differentiable Sparse Linear Solvers
Yu Wang (Massachusetts General Hospital), Bruce Fischl (Massachusetts General Hospital)
OptimizationComputational EfficiencyImage
🎯 What it does: A sparse linear solver based on Schwarz-Schur inversion is proposed, which compresses the sparse Laplacian matrix into a batch dense tensor and utilizes GPU BLAS for parallel inversion, achieving a speedup of 100-1000 times on sparse systems with image sizes ranging from 513×513 to 2561×2561, and is compatible with automatic differentiation, allowing direct integration into deep learning pipelines.
sciLaMA: A Single-Cell Representation Learning Framework to Leverage Prior Knowledge from Large Language Models
Hongru Hu (University of California), Gerald Quon (University of California)
Representation LearningLarge Language ModelAuto EncoderBiomedical Data
🎯 What it does: In single-cell RNA sequencing analysis, the sciLaMA framework is proposed, utilizing gene static embeddings generated by a pre-trained multimodal large language model (LLM) and transforming them into context-aware single-cell and gene representations through a paired VAE (with cell and gene encoders and decoders);
SCISSOR: Mitigating Semantic Bias through Cluster-Aware Siamese Networks for Robust Classification
Shuo Yang (Technical University of Munich), Gjergji Kasneci (Technical University of Munich)
ClassificationTransformerContrastive LearningImageText
🎯 What it does: A method named SCISSOR is proposed, which is a bias removal approach based on Siamese networks, aimed at eliminating skewed clustering in the semantic embedding space to suppress shortcut learning.
Score as Action: Fine Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning
Hanyang Zhao (Columbia University), Wenpin Tang (Columbia University)
GenerationReinforcement LearningDiffusion modelScore-based ModelImage
🎯 What it does: A continuous-time reinforcement learning framework is proposed, treating the score function of the diffusion model as an action for fine-tuning;
Score Matching with Missing Data
Josh Givens (University of Bristol), Henry Reeve
Score-based ModelTabularTime SeriesFinance Related
🎯 What it does: This study investigates how to learn the gradient (score) of a distribution under partially missing data and proposes two adaptation methods.
Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport
Mingyang Sun (Zhejiang University), Donglin Wang (Westlake University)
Robotic IntelligenceReinforcement LearningDiffusion model
🎯 What it does: This paper proposes the OTPR method, which combines diffusion policies and reinforcement learning through optimal transport theory, achieving end-to-end training from expert demonstrations to online fine-tuning;
Score-based Pullback Riemannian Geometry: Extracting the Data Manifold Geometry using Anisotropic Flows
Willem Diepeveen (University of California), Carola-Bibiane Schönlieb
Explainability and InterpretabilityRepresentation LearningScore-based ModelFlow-based ModelAuto EncoderImageTabular
🎯 What it does: This paper proposes a score function-based pullback Riemannian geometry framework that can learn interpretable low-dimensional representations from unimodal data distributions and provide closed-form geodesics.
Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions
Tejas Jayashankar (Massachusetts Institute of Technology), Gregory W. Wornell
GenerationData SynthesisDiffusion modelScore-based ModelGenerative Adversarial NetworkImage
🎯 What it does: Proposes Score-of-Mixture Training (SMT), which directly trains first-order generative models by minimizing the α-skew Jensen-Shannon divergence, and can be implemented without pre-trained diffusion models;
scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data
Olga Ovcharenko (BIFOLD & TU Berlin), Valentina Boeva (ETH Zurich)
Representation LearningData-Centric LearningContrastive LearningBiomedical DataBenchmark
🎯 What it does: Constructed scSSL-Bench, a systematic evaluation of 19 self-supervised learning (SSL) methods on single-cell data;
SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations
Grigory Bartosh (University of Amsterdam), Christian A. Naesseth (University of Amsterdam)
Score-based ModelVideoStochastic Differential Equation
🎯 What it does: Proposes SDE Matching, an implicit SDE training framework that does not require numerical simulation, is parallelizable, and has memory and time complexity of O(1), enabling scalable training of Latent SDE models.
SDMG: Smoothing Your Diffusion Models for Powerful Graph Representation Learning
Junyou Zhu (Potsdam Institute for Climate Impact Research), Frank Hellmann (Potsdam Institute for Climate Impact Research)
Representation LearningGraph Neural NetworkDiffusion modelGraph
🎯 What it does: Designed the SDMG (Smooth Diffusion Model for Graphs) framework, which utilizes a diffusion probability model for unsupervised representation learning on graph data, and enhances representation quality through a low-frequency encoder and multi-scale smooth loss.
SDP-CROWN: Efficient Bound Propagation for Neural Network Verification with Tightness of Semidefinite Programming
Hong-Ming Chiu (University of Illinois), Richard Y. Zhang (University of Illinois)
OptimizationComputational EfficiencyConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: Proposes the SDP-CROWN framework, which combines SDP and linear bound propagation to achieve scalable robustness verification of neural networks under ℓ₂ error;
SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space
Xupeng Zhu (Northeastern University), Jane Shi (Amazon Robotics)
Robotic IntelligenceGraph Neural NetworkReinforcement LearningDiffusion modelPoint Cloud
🎯 What it does: This paper proposes a SE(3) equivariant diffusion policy named Spherical Diffusion Policy (SDP) for learning robot manipulation strategies that can adapt to different 3D scenes.
SEAD: Unsupervised Ensemble of Streaming Anomaly Detectors
Saumya Gaurang Shah (Amazon Web Services), Vikramank Singh (Amazon Web Services)
Anomaly DetectionMixture of ExpertsTime Series
🎯 What it does: This paper proposes SEAD, an online unsupervised anomaly detection model selection and ensemble method;
Secant Line Search for Frank-Wolfe Algorithms
Deborah Hendrych (Zuse Institute Berlin), David Martínez-Rubio (Carlos III University of Madrid)
OptimizationBenchmark
🎯 What it does: A new line search strategy called Secant Line Search (SLS) is proposed for the Frank-Wolfe algorithm, which dynamically determines the step size by transforming the line search problem into a root-finding problem.
SecEmb: Sparsity-Aware Secure Federated Learning of On-Device Recommender System with Large Embedding
Peihua Mai (National University of Singapore), Yan Pang (National University of Singapore)
Recommendation SystemFederated LearningSafty and PrivacyComputational EfficiencyTabularSequential
🎯 What it does: A sparse encrypted federated recommendation system protocol named SecEmb is proposed, which utilizes point functions and function secret sharing (FSS) to ensure that users only download item embeddings they have interacted with, while securely aggregating sparse gradients on the server side, ensuring that the server cannot obtain any user's rated item indices or gradient information.
SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding
Woohyeon Park (Seoul National University), Jaeyoung Do (Seoul National University)
RecognitionObject DetectionTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: By selectively and contrastively decoding multi-scale visual information, the perceptual illusion of Vision-Language models is reduced, and the accuracy of object recognition and answering is improved.
Securing Equal Share: A Principled Approach for Learning Multiplayer Symmetric Games
Jiawei Ge (Princeton University), Chi Jin (Princeton University)
OptimizationReinforcement Learning from Human FeedbackReinforcement Learning
🎯 What it does: This paper studies multi-player symmetric constant-sum games and proposes a theoretical framework and implementable algorithms that ensure learners achieve fair payoffs (equal shares) under specific conditions.
SeedLoRA: A Fusion Approach to Efficient LLM Fine-Tuning
Yong Liu (National University of Singapore), Yang You (National University of Singapore)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: We propose SeedLoRA, a two-stage fusion method for LoRA models trained with different random seeds on the same task, aimed at narrowing the performance gap between LoRA and full fine-tuning.
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning
Jinpeng Chen (City University of Hong Kong), Sam Kwong (Lingnan University)
RecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningMultimodality
🎯 What it does: A forgetting elimination framework SEFE for multimodal continual instruction tuning is proposed, which separates and simultaneously suppresses superficial forgetting and essential forgetting.
Segment Anyword: Mask Prompt Inversion for Open-Set Grounded Segmentation
Zhihua Liu (University of Leicester), Chen Jin (AstraZeneca)
SegmentationPrompt EngineeringDiffusion modelImageMultimodality
🎯 What it does: We propose Segment Anyword, a completely training-free multimodal segmentation framework that utilizes cross-attention maps from a frozen diffusion model to generate visual concept prompts, thereby driving segmenters like SAM to achieve open-set language-guided segmentation.
Selective Preference Aggregation
Shreyas Kadekodi (University of California San Diego), Berk Ustun (University of California San Diego)
Recommendation SystemOptimizationTabular
🎯 What it does: Selective preference aggregation is proposed, expressing collective preferences as hierarchical rankings, allowing comparisons to stop on pairs of items with disagreements, ensuring that any comparable pair meets the preferences of at least (1-τ) proportion of users.
Selective Prompt Anchoring for Code Generation
Yuan Tian (Purdue University), Tianyi Zhang (Purdue University)
GenerationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This study investigates the phenomenon of attention decay in the code generation process of large language models and proposes the Selective Prompt Anchoring (SPA) method, which enhances the impact of keywords in user prompts on model attention to improve generation quality.
Selective Response Strategies for GenAI
Boaz Taitler (Technion Israel Institute of Technology), Omer Ben-Porat (Technion Israel Institute of Technology)
GenerationData SynthesisReinforcement Learning
🎯 What it does: This paper proposes a new strategy called selective response, aimed at addressing the issue of potentially inaccurate answers generated by generative artificial intelligence (GenAI) when dealing with emerging topics and technologies, thereby encouraging users to return to human-driven platforms (such as Stack Overflow) to generate high-quality data.
Self-Bootstrapping for Versatile Test-Time Adaptation
Shuaicheng Niu (Nanyang Technological University), Zhiqi Shen (Nanyang Technological University)
ClassificationObject DetectionSegmentationDomain AdaptationImage
🎯 What it does: A universal self-guided adaptive (SPA) framework is proposed, which can perform model adaptation on image, object, and pixel-level tasks without source data or changes in the training process.
Self-Consistency Preference Optimization
Archiki Prasad (University of North Carolina Chapel Hill), Jane Yu
OptimizationLarge Language ModelReinforcement LearningText
🎯 What it does: This study investigates the use of Self-Consistent Preference Optimization (SCPO) for self-training large language models under the condition of no gold labels to enhance performance on complex reasoning tasks.
Self-Consuming Generative Models with Adversarially Curated Data
Xiukun Wei (Ohio State University), Xueru Zhang (Ohio State University)
GenerationOptimizationAdversarial AttackDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: This paper studies the evolution and vulnerability of models encountered during adversarial data curation in self-consumption generative model training loops, and proposes an attack algorithm.
Self-cross Feature based Spiking Neural Networks for Efficient Few-shot Learning
Qi Xu (Dalian University of Technology), Qiang Zhang (Dalian University of Technology)
ClassificationMeta LearningSpiking Neural NetworkContrastive LearningImage
🎯 What it does: A few-shot learning framework based on Spiking Neural Networks (SNN), called SSCF, is proposed, which combines a self-feature extraction module and a cross-feature comparison module, and uses TET and InfoNCE loss for end-to-end training.
Self-Discriminative Modeling for Anomalous Graph Detection
Jinyu Cai (National University of Singapore), Jicong Fan (Chinese University of Hong Kong)
Anomaly DetectionGraph Neural NetworkGenerative Adversarial NetworkContrastive LearningGraph
🎯 What it does: This paper proposes a self-adversarial pseudo-anomaly graph generation and discrimination framework that implements layer anomaly detection using a deep network trained solely on positive samples.
Self-Disentanglement and Re-Composition for Cross-Domain Few-Shot Segmentation
Jintao Tong (Huazhong University of Science and Technology), Ruixuan Li (Huazhong University of Science and Technology)
SegmentationDomain AdaptationTransformerImage
🎯 What it does: A self-separation and recombination method is proposed to address the feature entanglement problem in cross-domain few-shot segmentation (CD-FSS), utilizing the natural hierarchical structure of Vision Transformer (ViT) to split features and recombine them after cross-layer comparison to obtain segmentation results with better generalization ability.
Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI
Julien Pourcel (Inria), Pierre-Yves Oudeyer (Inria)
OptimizationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: The SOAR framework is proposed, which continuously improves the model during the search by embedding LLM into the evolutionary search loop, achieving program synthesis on the ARC task.
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
Nayoung Lee (University of Wisconsin Madison), Dimitris Papailiopoulos (Microsoft)
TransformerText
🎯 What it does: Using a standard Transformer without changing the architecture, the model gradually achieves generalization from easy to difficult and from short to long by self-generating training data and iterative training.
Self-Organizing Visual Prototypes for Non-Parametric Representation Learning
Thalles Silva (University of Campinas), Adín Ramírez Rivera (University of Oslo)
Object DetectionSegmentationRetrievalRepresentation LearningTransformerContrastive LearningImage
🎯 What it does: Proposes the Self-Organizing Prototypes (SOP) method, which constructs dynamic prototypes using non-parametric Support Embedding (SE) for unsupervised visual feature learning, and designs the SOP-Masked Image Modeling (SOP-MIM) task.
Self-Play $Q$-Learners Can Provably Collude in the Iterated Prisoner's Dilemma
Quentin Bertrand (Université Jean Monnet), Gauthier Gidel (Université de Montréal)
Reinforcement Learning
🎯 What it does: This paper proves that in the iterated prisoner's dilemma with self-play, the standard ε-greedy Q-learning algorithm can gradually converge from an always-defecting initial strategy to the cooperative Pavlov (win-stay, lose-shift) strategy, leading to algorithmic cooperation (i.e., 'collusion');
Self-supervised Adversarial Purification for Graph Neural Networks
Woohyun Lee (Sungkyunkwan University), Hogun Park (Sungkyunkwan University)
OptimizationAdversarial AttackGraph Neural NetworkAuto EncoderGraph
🎯 What it does: A self-supervised adversarial purification framework is proposed, decoupling robustness and accuracy, and a GPR-GAE graph autoencoder is designed as a dedicated purifier.
Self-Supervised Learning of Intertwined Content and Positional Features for Object Detection
Kang-Jun Liu (Tohoku University), Takayuki Okatani (Tohoku University)
Object DetectionSegmentationTransformerContrastive LearningImage
🎯 What it does: A self-supervised learning method specifically designed for object detection and instance segmentation is proposed, using Vision Transformer as the backbone network, focusing on learning features that integrate content and positional information.
Self-supervised Masked Graph Autoencoder via Structure-aware Curriculum
Haoyang Li (Tsinghua University), Wenwu Zhu (Tsinghua University)
Representation LearningGraph Neural NetworkAuto EncoderGraph
🎯 What it does: A curriculum learning-based masked graph autoencoder (Cur-MGAE) is proposed, which utilizes difficulty measurement and self-paced scheduling for unsupervised node representation learning.
Self-Supervised Transformers as Iterative Solution Improvers for Constraint Satisfaction
Yudong Xu, Elias Boutros Khalil
OptimizationTransformerGraph
🎯 What it does: A self-supervised framework based on Transformer, called ConsFormer, is proposed for iteratively improving solutions to Constraint Satisfaction Problems (CSP).
SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models
Yung-Sung Chuang (Massachusetts Institute of Technology), Wen-tau Yih (Meta FAIR)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: Proposes the SelfCite method, which uses the context ablation probability of the LLM itself as a reward to generate more accurate fine-grained sentence-level citations;
Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning
Run He (South China University of Technology), Huiping Zhuang (South China University of Technology)
ClassificationRepresentation LearningConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A method called DPCR is proposed for class-incremental learning without sample examples, addressing the issues of semantic drift and decision bias.
Semantics-aware Test-time Adaptation for 3D Human Pose Estimation
Qiuxia Lin (National University of Singapore), Angela Yao (National University of Singapore)
Pose EstimationDomain AdaptationVision Language ModelVideo
🎯 What it does: A semantic-guided test-time adaptation (TTA) framework is proposed, using action-text alignment models like MotionCLIP for adaptive 3D human pose estimation, addressing prediction ambiguity caused by occlusion, truncation, and depth uncertainty.
Semi-Supervised Blind Quality Assessment with Confidence-quantifiable Pseudo-label Learning for Authentic Images
Yan Zhong (Peking University), Tingting Jiang (Peking University)
OptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A semi-supervised BIQA framework called CPL-IQA is proposed, which utilizes entropy minimization for label transformation and confidence quantifiable pseudo-label learning to evaluate the quality of real distorted images.
SEMU: Singular Value Decomposition for Efficient Machine Unlearning
Marcin Sendera (Jagiellonian University), Dawid Damian Rymarczyk
ClassificationGenerationComputational EfficiencyData-Centric LearningDiffusion modelImage
🎯 What it does: This paper proposes a method that utilizes Singular Value Decomposition (SVD) to project model gradients, selecting the most important weight subspace and fine-tuning only less than 1% of the parameters, thereby achieving efficient forgetting of specific data or concepts.
SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models
Cansu Sancaktar (Autonomous Learning, University of Tübingen), Georg Martius (Max Planck Institute for Intelligent Systems)
Robotic IntelligenceTransformerReinforcement LearningVision Language ModelWorld ModelImageMultimodality
🎯 What it does: The SENSEI framework is proposed, which utilizes visual language models (VLM) to extract 'interestingness' rewards from initial self-exploration data, allowing world model-based RL agents to learn this reward for semantic self-exploration.
Separating Knowledge and Perception with Procedural Data
Adrian Rodriguez-Munoz (Massachusetts Institute of Technology), Antonio Torralba (Massachusetts Institute of Technology)
ClassificationSegmentationRetrievalTransformerContrastive LearningImageBiomedical Data
🎯 What it does: This study achieves unsupervised transfer and efficient de-learning/privacy control for visual tasks by training visual feature embeddings using procedural data and combining them with visual memory (KNN retrieval).
SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
Guoxuan Chen (Huawei Noah's Ark Lab), Chao Huang (University of Hong Kong)
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes SepLLM, a sparse attention mechanism that treats separator tokens as compression points for segment information, retaining only the first word, adjacent words, and separators, significantly accelerating the inference and training of LLMs.
SERENA: A Unified Stochastic Recursive Variance Reduced Gradient Framework for Riemannian Non-Convex Optimization
Yan Liu (Nankai University), Ruxin Wang (Shenzhen Institutes of Advanced Technology)
OptimizationTabular
🎯 What it does: This paper proposes the SRVRG estimator, the SRGE unified estimator, and the SERENA framework, extending variance reduction methods to Riemannian geometric non-convex optimization, and provides theoretical convergence analysis and experimental validation.
Set Valued Predictions For Robust Domain Generalization
Ron Tsibulsky (Tel Aviv University), Uri Shalit (Technion)
Domain AdaptationContrastive LearningImageBenchmark
🎯 What it does: This paper proposes a domain generalization method based on ensemble prediction to achieve robust performance on unseen domains.
Settling the Maximin Share Fairness for Scheduling among Groups of Machines
Bo Li (Hong Kong Polytechnic University), Xing Shiji
Optimization
🎯 What it does: This paper studies the maximum-minimum share (MMS) fair allocation problem in scheduling among machine groups, and provides polynomial-time approximation algorithms for both unrelated and identical machine groups.
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
Tianzhe Chu (Hong Kong University), Yi Ma (University of Alberta)
RecognitionSupervised Fine-TuningReinforcement LearningImageText
🎯 What it does: This paper compares and analyzes the memory and generalization performance of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) in the later training of foundational models, using the arithmetic reasoning card game GeneralPoints and the visual navigation task V-IRL.
SGD Jittering: A Training Strategy for Robust and Accurate Model-Based Architectures
Peimeng Guan (Georgia Institute of Technology), Mark A. Davenport (Georgia Institute of Technology)
RestorationOptimizationAdversarial AttackImageMagnetic Resonance Imaging
🎯 What it does: A strategy is proposed to inject SGD jitter noise during model-based architecture (MBA) training to enhance the robustness and generalization accuracy of inverse problem solving.
ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
Hanshi Sun (ByteDance), Beidi Chen (Carnegie Mellon University)
TransformerLarge Language ModelText
🎯 What it does: We propose SHADOWKV, a high-throughput long-context LLM inference system that utilizes low-rank key caching on GPU and offloads values to CPU;
Sharp Generalization for Nonparametric Regression by Over-Parameterized Neural Networks: A Distribution-Free Analysis in Spherical Covariate
Yingzhen Yang (Arizona State University)
Tabular
🎯 What it does: This paper studies the sharp generalization bounds of over-parameterized two-layer neural networks trained by gradient descent in non-parametric regression, particularly in a distribution-independent analysis under spherical covariates.
Sharp Optimality of Simple, Plug-in Estimation of the Fisher Information of a Smoothed Density
Subhodh Kotekal (University of Chicago)
🎯 What it does: The study estimates the Fisher information of the density after Gaussian smoothing under the condition of given independent and identically distributed data, and provides precise minimax error upper bounds for different noise levels.
SHARP-Distill: A 68× Faster Recommender System with Hypergraph Neural Networks and Language Models
Saman Forouzandeh (RMIT University), Mahdi Jalili (RMIT University)
Recommendation SystemComputational EfficiencyKnowledge DistillationGraph Neural NetworkContrastive LearningText
🎯 What it does: This paper proposes SHARP-Distill, a recommendation system based on a teacher-student knowledge distillation framework that integrates hypergraph neural networks and pre-trained language models to enhance recommendation quality and significantly accelerate inference speed.
SHE: Streaming-media Hashing Retrieval
Ruitao Pu (Sichuan University), Yuan Sun (Sichuan University)
RetrievalMultimodality
🎯 What it does: A Streaming-media Hashing Retrieval (SHE) method is proposed, supporting parallel learning and retrieval of streaming multimodal data.
SHIELD: Multi-task Multi-distribution Vehicle Routing Solver with Sparsity and Hierarchy
Yong Liang Goh (National University of Singapore), Wee Sun Lee (National University of Singapore)
OptimizationTransformerReinforcement LearningMixture of ExpertsGraph
🎯 What it does: A unified neural solver SHIELD suitable for multi-task multi-distribution vehicle routing planning is proposed, which can maintain efficiency and robustness under different tasks and geographical distributions.
ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning
Zhaorun Chen (University of Chicago), Bo Li (University of Illinois at Urbana-Champaign)
Safty and PrivacyExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelAgentic AITextBenchmark
🎯 What it does: A guardian agent named SHIELDAGENT has been constructed and deployed, capable of extracting verifiable rules from policy documents, constructing action-based probabilistic rule circuits, and utilizing logical reasoning and formal verification to conduct security checks and interceptions on the action trajectories of other LLM agents.
Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellency
Michael Kirchhof (Apple), marco cuturi
GenerationData SynthesisDiffusion modelImageMultimodalityStochastic Differential Equation
🎯 What it does: This paper proposes SPELL (Sparse Repellency), a sparse repulsion mechanism trained post-hoc, which can suppress overlap with a reference image set during the generation process of diffusion models from text to image or category to image, thereby enhancing diversity and achieving image protection.
Shifting Time: Time-series Forecasting with Khatri-Rao Neural Operators
Srinath Dama (University of Toronto), Prasanth B. Nair (University of Toronto)
Time SeriesSequential
🎯 What it does: This paper proposes a continuous time displacement operator and Khatri-Rao neural operator for learning continuous predictions of time series and spatiotemporal data.
Shortcut-connected Expert Parallelism for Accelerating Mixture of Experts
Weilin Cai (Hong Kong University of Science and Technology), Jiayi Huang (Hong Kong University of Science and Technology)
OptimizationComputational EfficiencyTransformerMixture of ExpertsImageText
🎯 What it does: Proposes the Shortcut-connected Expert Parallelism (ScMoE) architecture, which combines an adaptive overlapping parallel strategy to significantly accelerate the distributed training and inference of sparse gated Mixture-of-Experts models.
Should Decision-Makers Reveal Classifiers in Online Strategic Classification?
Han Shao (Harvard University), Kunhe Yang (University of California)
Finance Related
🎯 What it does: This paper studies whether decision-makers should disclose classifiers in online strategic classification, particularly in cases where agents may manipulate features to obtain favorable predictions.
Sidechain conditioning and modeling for full-atom protein sequence design with FAMPNN
Talal Widatalla (Stanford University), Possu Huang
Protein Structure PredictionGraph Neural NetworkDiffusion modelBiomedical Data
🎯 What it does: A full-atom protein sequence design method called FAMPNN is proposed, which can simultaneously predict the amino acid sequence identity and the three-dimensional coordinates of side chains within the same model, and achieves joint generation of sequences and side chains through an iterative masked language model.
Signed Laplacians for Constrained Graph Clustering
John Stewart Fabila Carrasco (University of Edinburgh), He Sun (University of Edinburgh)
OptimizationComputational EfficiencyGraph Neural NetworkGraphTime Series
🎯 What it does: A generalized eigenvalue method based on the signed Laplacian is proposed to solve the graph clustering problem with MUST-LINK and CANNOT-LINK constraints, along with the corresponding Cheeger-type inequality.
Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generation
Yoann Boget (University of Applied Sciences and Arts of Western Switzerland)
GenerationData SynthesisDrug DiscoveryGraph Neural NetworkDiffusion modelGraphBiomedical Data
🎯 What it does: A simplified discrete diffusion framework called Simple Iterative Denoising (SID) is proposed, which eliminates the cumulative error in traditional discrete diffusion by assuming conditional independence of intermediate noise states. Based on this, a Critic mechanism is added to dynamically adjust the re-noising probability of each element, forming Critical Iterative Denoising (CID).
Simple Path Structural Encoding for Graph Transformers
Louis Airale (University of Trento), Roberto Passerone (University of Trento)
Drug DiscoveryGraph Neural NetworkTransformerGraph
🎯 What it does: A simple path counting-based edge encoding method SPSE is proposed to improve the structural encoding of graph Transformers.
Simple Policy Optimization
Zhengpeng Xie (Hong Kong University of Science and Technology), Renjing Xu (Hong Kong University of Science and Technology)
OptimizationReinforcement LearningSequential
🎯 What it does: A new unconstrained first-order policy optimization algorithm SPO is proposed, which improves the ratio clipping loss of PPO and combines the convergence guarantees of TRPO.
Simple Randomized Rounding for Max-Min Eigenvalue Augmentation
Jourdain Lamperski (University of Pittsburgh), Oleg Prokopyev
OptimizationStochastic Differential Equation
🎯 What it does: For the maximum-minimum eigenvalue increment problem of a given symmetric positive semidefinite matrix M and several increment matrices A1,…,Am, a simple randomized rounding algorithm is proposed and analyzed. This algorithm, under the condition that k < n, obtains integer solutions by sampling from the SDP relaxation solution, proving that a constant factor approximation can be achieved when the optimal increment is sufficiently large.
SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning
Tianjian Li (Johns Hopkins University), Daniel Khashabi (Johns Hopkins University)
Recommendation SystemOptimizationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper systematically compares the on-policy and off-policy data used in aligning language models, finding that both have complementary advantages across different task types, and proposes a simple data mixing method called SIMPLEMIX.
Simplicity Bias and Optimization Threshold in Two-Layer ReLU Networks
Etienne Boursier (INRIA), Nicolas Flammarion (EPFL)
OptimizationTabular
🎯 What it does: This paper explores the generalization ability of over-parameterized models in two-layer ReLU networks, particularly the phenomenon where the model shifts from a global minimum to simpler solutions when the number of training samples exceeds a certain optimization threshold.
Simplifying DINO via Coding Rate Regularization
Ziyang Wu (University of California Berkeley), Yi Ma (University of California Berkeley)
Object DetectionSegmentationRepresentation LearningTransformerContrastive LearningImageVideo
🎯 What it does: By adding a coding rate regularization term to the training objectives of DINO and DINOv2, and removing a large number of empirical hyperparameters and complex post-processing, two simpler and more stable self-supervised pre-training models, SimDINO and SimDINOv2, are obtained.
Simultaneous Multi-Robot Motion Planning with Projected Diffusion Models
Jinhao Liang (University of Virginia), Ferdinando Fioretto (University of Virginia)
OptimizationRobotic IntelligenceDiffusion modelScore-based ModelTabularBenchmark
🎯 What it does: A multi-robot motion planning method named SMD (Simultaneous MRMP Diffusion) is proposed, which combines constrained optimization with a diffusion model. During the sampling process, projection is used to ensure that the generated trajectories meet collision avoidance and kinematic constraints.
Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery
Mateusz Olko (University of Warsaw), Piotr Miłoś (University of Warsaw)
Graph
🎯 What it does: A systematic evaluation of the performance of causal discovery methods using neural networks under limited data and the proposal of the λ-strong confidence measure.
SING: Spatial Context in Large Language Model for Next-Gen Wearables
Ayushi Mishra (University of Maryland), Nirupam Roy (University of Maryland)
Convolutional Neural NetworkLarge Language ModelAudio
🎯 What it does: A system architecture that integrates spatial audio perception into large language models, specifically designed for wearable devices;
SITCOM: Step-wise Triple-Consistent Diffusion Sampling For Inverse Problems
Ismail Alkhouri, Rongrong Wang (Michigan State University)
RestorationOptimizationDiffusion modelImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes an optimized sampler named SITCOM, which utilizes diffusion models to achieve more efficient reconstruction in inverse problem solving through three consistency constraints.
SK-VQA: Synthetic Knowledge Generation at Scale for Training Context-Augmented Multimodal LLMs
Xin Su (Intel Labs), Phillip Howard (Thoughtworks)
Data SynthesisRetrievalTransformerLarge Language ModelImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: A synthetic dataset (SK-VQA) containing over 2 million visual question-answer pairs and corresponding contextual documents has been constructed, and it is used to train a multimodal large language model compatible with retrieval-augmented generation.
Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation
Tianyi Zhang (Rice University), Anshumali Shrivastava (Rice University)
CompressionDomain AdaptationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningTextMultimodality
🎯 What it does: This paper proposes a method called SketchTune for compressing LLM weights through learned sketching and directly fine-tuning, balancing model compression and adaptation, eliminating the low-rank assumption and multi-path computation of traditional PEFT.
SketchDNN: Joint Continuous-Discrete Diffusion for CAD Sketch Generation
Sathvik Reddy Chereddy (Miami-Oxford University), John Femiani (Miami-Oxford University)
GenerationData SynthesisTransformerDiffusion modelGraph
🎯 What it does: A joint continuous-discrete diffusion model called SketchDNN has been developed for generating CAD sketches, capable of handling both geometric parameters and category labels simultaneously.
SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization
Runsheng Bai (Massachusetts Institute of Technology), qiang liu
CompressionOptimizationTransformerLarge Language ModelText
🎯 What it does: A post-training, weight-only quantization method called SKIM is proposed for compressing large language models at arbitrary bit widths.
Skip the Equations: Learning Behavior of Personalized Dynamical Systems Directly From Data
Krzysztof Kacprzyk (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
Explainability and InterpretabilityDrug DiscoveryTabularTime SeriesBiomedical DataOrdinary Differential Equation
🎯 What it does: EPISODE is proposed, utilizing a direct semantic modeling approach to predict multidimensional personalized dynamic systems, directly outputting the semantic representation of trajectories from static features without the need to first solve ODEs;
SkipGPT: Each Token is One of a Kind
Anhao Zhao (Eastern Institute of Technology), Xiaoyu Shen (Eastern Institute of Technology)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes SkipGPT, a framework based on dynamic layer pruning that achieves adaptive inference by utilizing the computational demands of each token.
SKOLR: Structured Koopman Operator Linear RNN for Time-Series Forecasting
Yitian Zhang (McGill University), Mark Coates (McGill University)
Recurrent Neural NetworkTime Series
🎯 What it does: This paper proposes a time series forecasting framework SKOLR based on the Koopman operator theory, establishing an equivalence with linear RNNs and achieving efficient Koopman approximation through learnable frequency decomposition and an MLP encoder.
Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient Text-to-Image Generation
Hoigi Seo (Seoul National University), Se Young Chun (Seoul National University)
GenerationCompressionTransformerVision Language ModelImageText
🎯 What it does: Proposes the Skrr method, which implements hierarchical skipping and reuse for text encoders to achieve compression.
Sleeping Reinforcement Learning
Simone Drago (Politecnico di Milano), Alberto Maria Metelli (Politecnico di Milano)
Reinforcement Learning
🎯 What it does: A new reinforcement learning paradigm called Sleeping Reinforcement Learning is proposed, which allows the set of available actions to change during interaction with the environment.
Sliding Puzzles Gym: A Scalable Benchmark for State Representation in Visual Reinforcement Learning
Bryan Lincoln Marques de Oliveira, Luckeciano Carvalho Melo
Representation LearningReinforcement LearningAuto EncoderContrastive LearningImageBenchmark
🎯 What it does: This paper proposes the Sliding Puzzles Gym (SPGym), a scalable visual reinforcement learning benchmark that maps the classic 8-tile puzzle to image observations and allows for the adjustment of image pool size and grid dimensions, while keeping the environment dynamics unchanged, thereby isolating and systematically evaluating the capability of representation learning.
SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
Wei Huang (University of Hong Kong), XIAOJUAN QI
CompressionOptimizationTransformerLarge Language ModelText
🎯 What it does: This paper proposes SliM-LLM, a weight significance-driven grouped mixed-precision post-training quantization framework for compressing large language models.
SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression
Mohammad Mozaffari (University of Toronto), Maryam Mehri Dehnavi (NVIDIA Corporation)
CompressionTransformerLarge Language ModelText
🎯 What it does: This paper proposes a one-stop compression framework called SLIM, which integrates unified quantization, semi-structured sparsity, and low-rank adapters to achieve efficient compression of LLM weights.
SlimLLM: Accurate Structured Pruning for Large Language Models
Jialong Guo (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab)
TransformerLarge Language ModelText
🎯 What it does: Structured pruning of large language models (LLaMA series) is performed, removing attention heads and feedforward network channels, while compensating for performance loss using linear regression.
Slimming the Fat-Tail: Morphing-Flow for Adaptive Time Series Modeling
Tianyu Liu (Tsinghua University), Yuanlong Zhang (Tsinghua University)
Flow-based ModelTime SeriesFinance Related
🎯 What it does: This paper proposes Morphing-Flow (MoF), which utilizes a reversible piecewise spline transformation layer (Flow) and a test-time adaptive module (Morph) to dynamically normalize non-stationary, heavy-tailed time series while preserving extreme features.
SMART-PC: Skeletal Model Adaptation for Robust Test-Time Training in Point Clouds
Ali Bahri (ETS Montreal), Christian Desrosiers (ETS Montreal)
ClassificationDomain AdaptationPoint Cloud
🎯 What it does: This paper proposes SMART-PC, a gradient-free testing and training framework based on skeletons, which utilizes skeleton representations to achieve real-time adaptation for 3D point cloud classification.
Smooth Interpolation for Improved Discrete Graph Generative Models
Yuxuan Song (Tsinghua University), Wei-Ying Ma
GenerationData SynthesisDrug DiscoveryGraph Neural NetworkTransformerDiffusion modelFlow-based ModelGraph
🎯 What it does: This paper proposes Graph Bayesian Flow Networks (GraphBFN), a generative framework that smoothly transitions discrete graph structures in continuous probability matrix space.
Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences
Yunhong Lu (Zhejiang University), Min Zhang (Zhejiang University)
GenerationOptimizationReinforcement LearningDiffusion modelImage
🎯 What it does: The SmPO-Diffusion method is proposed, which aligns human preferences in text-to-image diffusion models through smoothing preference distributions and Renoise Inversion, significantly improving generation quality and reducing training costs.
SNS-Bench: Defining, Building, and Assessing Capabilities of Large Language Models in Social Networking Services
Hongcheng Guo (Beihang University), Zhoujun Li
Recommendation SystemTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: A social network service (SNS) large language model (LLM) evaluation benchmark named SNS-BENCH has been constructed and released, containing 8 tasks with a total of 6,658 questions; a systematic evaluation of over 25 public and closed-source LLMs has also been conducted.
Socialized Coevolution: Advancing a Better World through Cross-Task Collaboration
Xinjie Yao (Tianjin University), Qinghua Hu (Tianjin University)
ClassificationObject DetectionContrastive LearningImage
🎯 What it does: A Social Co-evolution (SC) framework and DISC module are proposed, utilizing dynamic interactions between multi-task models to enhance existing task performance and learn new tasks.
Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration
Qinglin Zhu (King's College London), Lin Gui (King's College London)
TransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Improving the reasoning process of large language models, the Soft Reasoning method is proposed, which injects Gaussian noise into the embedding space of the first token and utilizes Bayesian optimization to search for the optimal embedding, thereby guiding the generation of more accurate and coherent answers.
Softmax is not Enough (for Sharp Size Generalisation)
Petar Veličković (Google DeepMind), Razvan Pascanu (Google DeepMind)
RetrievalOptimizationTransformerText
🎯 What it does: This paper studies the dispersion characteristics of softmax as the input size increases, proving that softmax must disperse with more input items, leading to an inability to maintain the robustness of 'sharp' functions (such as maximum value retrieval). It proposes an entropy-based adaptive temperature method to alleviate this issue.