ICLR 2025 Papers — Page 32
International Conference on Learning Representations · 3704 papers
Spherical Tree-Sliced Wasserstein Distance
Hoang V. Tran, Tan Minh Nguyen
OptimizationComputational EfficiencyAuto EncoderContrastive LearningImage
🎯 What it does: A tree-cut Wasserstein distance based on spherical trees (STSW) is proposed to measure probability distributions on the sphere.
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
Fangyu Lei (University of Hong Kong), Tao Yu (University of Hong Kong)
Large Language ModelAgentic AITextBenchmark
🎯 What it does: The Spider 2.0 benchmark is proposed to evaluate language models in real enterprise-level text-to-SQL workflows.
SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking
Xingrun Xing (University of Chinese Academy of Sciences), Jiajun Zhang (Institute of Automation)
Spiking Neural NetworkLarge Language ModelText
🎯 What it does: Designed and implemented a scalable spiking neural network large language model, SpikeLLM, with parameters ranging from 7 to 70 billion, and proposed GIF neurons and the Optimal Brain Spiking (OBSpiking) framework to enhance spiking encoding efficiency and trainability.
Spiking Vision Transformer with Saccadic Attention
Shuai Wang (University of Electronic Science and Technology of China), Yang Yang (University of Electronic Science and Technology of China)
ClassificationObject DetectionSpiking Neural NetworkTransformerImage
🎯 What it does: A SNN-ViT that combines spiking neural networks and visual Transformers is proposed and implemented, and a Saccadic Spike Self-Attention (SSSA) mechanism is designed to address the matching issue between SNN and self-attention.
SpinQuant: LLM Quantization with Learned Rotations
Zechun Liu (Meta), Tijmen Blankevoort (Meta)
TransformerLarge Language ModelText
🎯 What it does: This paper presents SpinQuant, a method that eliminates outliers during quantization in large language models by learning rotation matrices, significantly improving the quantization accuracy of 4-bit weights, activations, and KV caches.
SplatFormer: Point Transformer for Robust 3D Gaussian Splatting
Yutong Chen (ETH Zurich), Siyu Tang (ETH Zurich)
TransformerGaussian SplattingPoint Cloud
🎯 What it does: This paper proposes a point Transformer model called SplatFormer, which is used to refine 3D Gaussian Splatting (3DGS) representations in a single pass, thereby improving the rendering quality of views outside the input perspective distribution (OOD).
SplineGS: Learning Smooth Trajectories in Gaussian Splatting for Dynamic Scene Reconstruction
Jihwan Yoon (Hanyang University), Minsik Lee (Hanyang University)
RestorationOptimizationGaussian SplattingPoint Cloud
🎯 What it does: This paper proposes a dynamic scene reconstruction method called SplineGS based on 3D Gaussian splatting, which utilizes Non-Uniform Rational B-Splines (NURBS) to learn the smooth trajectories of Gaussian clouds. It achieves high-quality and fast dynamic scene reconstruction through a linear combination of representative trajectories and multi-resolution hashing + MLP learning weights.
Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports
Yi Xu (Northeastern University), Yun Fu (Northeastern University)
GenerationTransformerAuto EncoderTime Series
🎯 What it does: A unified trajectory generation model called UniTraj is proposed, capable of handling various tasks such as trajectory prediction, completion, and spatiotemporal recovery in one go.
SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models
Haotian Xia (University of California, Irvine), Hanjie Chen (Rice University)
TransformerLarge Language ModelPrompt EngineeringVideoTextMultimodalityBenchmark
🎯 What it does: The SPORTU sports understanding benchmark is proposed, which includes two subsets: text and slow-motion videos, to evaluate the capabilities of multimodal large language models in rule reasoning, strategy understanding, and video perception.
Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
Dongyoung Kim (Korea Advanced Institute of Science and Technology), Jaehyung Kim (Yonsei University)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: By utilizing a minimal amount of manually labeled preference data, new preference samples are generated through self-supervision and self-correction, enhancing the consistency between LLMs and human preferences.
Spreading Out-of-Distribution Detection on Graphs
Daeho Um (Samsung Electronics), Yoonho Jung (Seoul National University)
Anomaly DetectionGraph Neural NetworkGraphBenchmark
🎯 What it does: This paper proposes a spreading OOD detection task on graph structures, constructs an evaluation benchmark based on epidemic propagation models, and introduces an energy distribution-based aggregation detector named EDBD.
Spurious Forgetting in Continual Learning of Language Models
Junhao Zheng (South China University of Technology), Qianli Ma (South China University of Technology)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the phenomenon of 'pseudo-forgetting' in large language models during continual learning, revealing that performance degradation primarily stems from task alignment failure rather than knowledge loss, and proposes a strategy of freezing lower-level parameters (Freeze) to alleviate this issue.
SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget
Zihao Wang (Peking University), Shaoduo Gan (Peking University)
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes a 2D compression algorithm called SQUEEZEATTENTION, which is based on hierarchical importance dynamic allocation of KV-cache. During the inference process, the influence of each attention layer is measured using cosine similarity, and the cache budget is reallocated according to importance.
SRSA: Skill Retrieval and Adaptation for Robotic Assembly Tasks
Yijie Guo (NVIDIA Corporation), Yashraj Narang (NVIDIA Corporation)
Robotic IntelligenceReinforcement LearningAuto EncoderPoint CloudMeshBenchmark
🎯 What it does: Design and implement the SRSA framework, which retrieves trained specialized assembly strategies for rapid adaptation, enhances convergence speed and stability through self-imitation learning, and achieves continuous learning and real-world deployment on the AutoMate benchmark.
SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes
Tony Alex (Surrey Institute for People Centred AI), Philip J B Jackson
Representation LearningConvolutional Neural NetworkTransformerContrastive LearningAudio
🎯 What it does: Proposed and implemented the SSLAM framework, which significantly improves the model's performance in multi-source audio scenarios by incorporating audio mixing (partial mixing) and source retention loss in self-supervised audio pre-training.
SSOLE: Rethinking Orthogonal Low-rank Embedding for Self-Supervised Learning
Lun Huang (Duke University), Guillermo Sapiro (Duke University)
Object DetectionRepresentation LearningConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: The Orthogonal Low-rank Embedding (OLE) method is migrated to Self-Supervised Learning (SSL), proposing the SSOLE framework to achieve both positive sample alignment and negative sample separation.
ST-GCond: Self-supervised and Transferable Graph Dataset Condensation
Beining Yang (University of Edinburgh), Jianxin Li (Guangxi Normal University)
Data SynthesisCompressionMeta LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper proposes a self-supervised transferable graph dataset compression framework ST-GCond, which generates synthetic graph datasets that are extremely small in size but rich in information while maintaining test performance.
Stabilized Neural Prediction of Potential Outcomes in Continuous Time
Konstantin Hess (Munich Center for Machine Learning), Stefan Feuerriegel (Munich Center for Machine Learning)
TabularBiomedical DataElectronic Health RecordsStochastic Differential Equation
🎯 What it does: A neural network method for estimating Conditional Average Potential Outcomes (CAPO) in continuous time, called SCIP-Net, is proposed, which can make accurate inferences in the presence of time-varying confounding.
Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation
Eliot Xing (Carnegie Mellon University), Jean Oh (Carnegie Mellon University)
Robotic IntelligenceReinforcement LearningPhysics Related
🎯 What it does: This paper proposes the Soft Analytic Policy Optimization (SAPO) algorithm and the Rewarped parallel differentiable multi-physics simulation platform, exploring the use of first-order analytical gradients in reinforcement learning for soft and rigid body interaction tasks.
Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning
Hung Le (Deakin University), Svetha Venkatesh (Deakin University)
Meta LearningRecurrent Neural NetworkReinforcement LearningAgentic AITime SeriesSequential
🎯 What it does: This paper proposes the Stable Hadamard Memory (SHM) — a memory augmentation network that utilizes the Hadamard product for memory calibration and updating, aimed at addressing memory management issues in RL under long time sequences and partially observable environments.
Stable Segment Anything Model
Qi Fan (Nanjing University), Chi-Keung Tang (Hong Kong University of Science and Technology)
SegmentationSupervised Fine-TuningImage
🎯 What it does: Stable-SAM is proposed to enhance segmentation stability under low-quality prompts (inaccurate boxes or sparse points) by adjusting the mask attention of SAM.
STAFF: Speculative Coreset Selection for Task-Specific Fine-tuning
Xiaoyu Zhang (Xi'an Jiaotong University), Yang Liu (Nanyang Technological University)
TransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: A task-specific LLM fine-tuning Coreset selection method named STAFF is proposed, which utilizes sibling small models to quickly estimate sample importance and validate on the target LLM, dynamically allocating selection budgets while balancing importance and diversity, significantly improving data efficiency and reducing selection costs.
STAMP: Scalable Task- And Model-agnostic Collaborative Perception
Xiangbo Gao (Texas A&M University), Zhengzhong Tu (Texas A&M University)
Object DetectionSegmentationAutonomous DrivingConvolutional Neural NetworkMultimodalityPoint Cloud
🎯 What it does: A scalable and task-agnostic multi-vehicle collaborative perception framework called STAMP is proposed, utilizing lightweight adapter-recovery modules to achieve cross-heterogeneous vehicle BEV feature alignment and fusion.
Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization
Zhitong Xu (University of Utah), Shandian Zhe (University of Utah)
OptimizationTabularBenchmark
🎯 What it does: This study investigates standard high-dimensional Bayesian optimization and finds that using the Matérn kernel performs excellently in high-dimensional scenarios. The theoretical analysis reveals the gradient vanishing problem of the SE kernel; a dimension-based length scale initialization strategy is proposed, significantly improving performance.
Standardizing Structural Causal Models
Weronika Ormaniec (ETH Zurich), Andreas Krause (ETH Zurich)
GenerationData Synthesis
🎯 What it does: This paper proposes a structural causal model with internal standardization for each variable during the generation process (iSCM) to eliminate artificial traces such as variance and correlation sorting (Var-sortability, R2-sortability) that arise in synthetic data from traditional SCM.
STAR: Stability-Inducing Weight Perturbation for Continual Learning
Masih Eskandar (Northeastern University), Jennifer Dy (Northeastern University)
OptimizationImage
🎯 What it does: Proposes the STAR regularization method, which minimizes the KL divergence under worst-case perturbations in the parameter space to enhance the output stability of the model on learned samples in continual learning.
STAR: Synthesis of Tailored Architectures
Armin W Thomas, Michael Poli (Liquid AI)
OptimizationNeural Architecture SearchTransformerLarge Language ModelText
🎯 What it does: The STAR framework is proposed, which constructs a hierarchical search space based on Linear Input Variable (LIV) systems, using evolutionary algorithms to automatically synthesize backbone structures of language models for multi-objective optimization regarding quality, parameter count, and inference cache.
Start Smart: Leveraging Gradients For Enhancing Mask-based XAI Methods
Buelent Uendes (Vrije Universiteit Amsterdam), Mark Hoogendoorn (Vrije Universiteit Amsterdam)
OptimizationExplainability and InterpretabilityRecurrent Neural NetworkImageTime Series
🎯 What it does: A gradient-based mask initialization method called StartGrad is proposed and implemented, significantly improving the optimization speed and final performance of mask-based XAI methods.
State Space Model Meets Transformer: A New Paradigm for 3D Object Detection
ChuXin Wang, Tianzhu Zhang (University of Science and Technology of China)
Object DetectionTransformerSimultaneous Localization and MappingPoint Cloud
🎯 What it does: A new 3D object detection paradigm called DEST is proposed, which combines state space models (SSM) and Transformers to address the performance limitations caused by fixed scene point features in the DETR method.
State Space Models are Provably Comparable to Transformers in Dynamic Token Selection
Naoki Nishikawa (University of Tokyo RIKEN AIP), Taiji Suzuki (University of Tokyo RIKEN AIP)
TransformerSequentialBenchmark
🎯 What it does: This paper demonstrates through theoretical and experimental research that embedding fully connected networks into state space models (SSM) can achieve dynamic token selection comparable to that of Transformers, and achieves performance on three types of tasks—input copying, associative recall, and non-parametric regression—that matches that of Transformers.
Statistical Advantages of Perturbing Cosine Router in Mixture of Experts
Huy Nguyen (University of Texas at Austin), Nhat Ho (University of Texas at Austin)
Domain AdaptationOptimizationTransformerMixture of ExpertsText
🎯 What it does: This study investigates the statistical convergence properties of the cosine router in the mixture of experts (MoE) model and proposes a 'perturbed cosine router' that adds noise to the L2 norm to improve the convergence speed of parameter and expert estimates.
Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs
Yuheng Zhang (University of Illinois Urbana-Champaign), Nan Jiang (University of Illinois Urbana-Champaign)
Reinforcement Learning
🎯 What it does: This paper studies the offline policy evaluation problem using history-dependent target policies in partially observable Markov decision processes (POMDPs), revealing the theoretical infeasibility of model-free methods in this context, and proposes a model-based approach using maximum likelihood estimation to achieve polynomial sample complexity.
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs
Peijie Dong (Hong Kong University of Science and Technology), Xiaowen Chu (Hong Kong University of Science and Technology)
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Perform structured binary compression with sub 1-bit precision on large language models, significantly reducing memory and computational load.
Stealthy Shield Defense: A Conditional Mutual Information-Based Approach against Black-Box Model Inversion Attacks
Tianqu Zhuang (Tsinghua University), Shu-Tao Xia (Tsinghua University)
ClassificationSafty and PrivacyAdversarial AttackContrastive LearningImage
🎯 What it does: This paper studies the defense against black-box model reverse attacks and proposes a post-processing method called Stealthy Shield Defense (SSD).
Steering Large Language Models between Code Execution and Textual Reasoning
Yongchao Chen (Massachusetts Institute of Technology), Chi Wang (Google DeepMind)
AI Code AssistantTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: This paper systematically evaluates the ability of large language models to switch between code execution and text reasoning across 14 diverse tasks, and proposes three improvement methods.
Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction
Jarrid Rector-Brooks (University of Montreal), Joey Bose
GenerationData SynthesisOptimizationReinforcement Learning from Human FeedbackProtein Structure PredictionReinforcement LearningDiffusion modelImageTextSequentialBiomedical Data
🎯 What it does: This paper proposes the DDPP framework, treating the tuning of the Masked Discrete Diffusion Model (MDM) as a sampling problem for the Bayesian posterior;
Steering Protein Family Design through Profile Bayesian Flow
Jingjing Gong (Institute of AI Industry Research), Wei-Ying Ma (Institute of AI Industry Research)
GenerationProtein Structure PredictionTransformerFlow-based ModelBiomedical Data
🎯 What it does: This paper proposes ProfileBayesian Flow Networks (ProfileBFN) for protein family design, which can utilize single-sequence training to generate diverse protein sequences that conform to family structure and function.
Stem-OB: Generalizable Visual Imitation Learning with Stem-Like Convergent Observation through Diffusion Inversion
Kaizhe Hu (Tsinghua University), Huazhe Xu (Tsinghua University)
Domain AdaptationRobotic IntelligenceReinforcement LearningDiffusion modelImage
🎯 What it does: Utilizing the reverse process of a pre-trained diffusion model for preprocessing visual observations (Stem-OB) to suppress low-level visual differences, retain high-order semantic structures, and enhance the model's generalization ability in unseen environments during visual imitation learning.
Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlo
Shengyu Feng (Carnegie Mellon University), Yiming Yang (Carnegie Mellon University)
OptimizationLarge Language ModelContrastive LearningTextBenchmark
🎯 What it does: A multi-step reasoning verification method based on Twisted Sequential Monte Carlo (TSMC) is proposed, which utilizes value function-guided sampling and resampling during the generation process, significantly improving reasoning quality and sampling efficiency.
Stiefel Flow Matching for Moment-Constrained Structure Elucidation
Austin Henry Cheng, Alan Aspuru-Guzik
GenerationDrug DiscoveryGraph Neural NetworkFlow-based ModelGraph
🎯 What it does: A generative model for flow matching on the Stiefel manifold has been developed to accurately recover three-dimensional molecular structures from molecular formulas and moments of inertia.
Stochastic Bandits Robust to Adversarial Attacks
Xuchuang Wang (University of Massachusetts Amherst), Mohammad Hajiesmaili (University of Massachusetts Amherst)
OptimizationAdversarial AttackReinforcement Learning from Human FeedbackTabular
🎯 What it does: This paper studies the stochastic multi-armed bandit under strong attack models and proposes robust algorithms that can achieve approximately optimal performance under known or unknown attack budgets.
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
Dimitris Oikonomou (Johns Hopkins University), Nicolas Loizou (Johns Hopkins University)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: Three stochastic Heavy Ball methods based on Polyak-style adaptive step sizes are proposed, along with a proof of convergence.
Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation
Chenyu Zhang (Massachusetts Institute of Technology), Xuan Di (Columbia University)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningSequential
🎯 What it does: A unified parameter stochastic semi-gradient descent (SemiSGD) algorithm is proposed, which directly updates both the policy and the population distribution simultaneously, thereby eliminating the back-and-forth loop of traditional fixed point iteration (FPI), suitable for continuous state-action spaces.
Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifold
Hoang Phuc Hau Luu (University of Helsinki), Arto Klami (University of Helsinki)
OptimizationTabular
🎯 What it does: A variance-reduced estimator for Gaussian variational inference on the Bures-Wasserstein manifold is proposed, significantly reducing the noise of single-sample Monte Carlo gradients, thereby achieving faster and more stable optimization.
StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spaces
Kyeongmin Yeo (KAIST), Minhyuk Sung (KAIST)
GenerationData SynthesisDiffusion modelScore-based ModelImageMesh
🎯 What it does: A zero-shot method called StochSync is proposed, which generates images in arbitrary spaces (such as 360° panoramas and 3D mesh surfaces) using a pre-trained diffusion model.
STORM: Spatio-TempOral Reconstruction Model For Large-Scale Outdoor Scenes
Jiawei Yang (University of Southern California), Marco Pavone (Stanford University)
Autonomous DrivingOptimizationTransformerGaussian SplattingOptical FlowImageVideo
🎯 What it does: We propose STORM, a self-supervised 4D scene reconstruction model based on Transformer, which can directly infer the geometry, appearance, and motion of dynamic scenes from sparse multi-time camera images by aggregating 3D Gaussian particles and applying time-varying transformations, without the need for per-scene optimization or additional motion supervision.
Storybooth: Training-Free Multi-Subject Consistency for Improved Visual Storytelling
Jaskirat Singh (Australian National University), Michael F Cohen
GenerationData SynthesisDiffusion modelImageTextMultimodality
🎯 What it does: A training and optimization-free multi-role consistency text-to-image generation method called StoryBooth is proposed to generate visual storyboards, enhancing the consistency of multiple characters and fine-grained details.
Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs
Shane Bergsma (Cerebras Systems), Joel Hestness (Cerebras Systems)
OptimizationTransformerLarge Language ModelText
🎯 What it does: The paper compares various learning rate schedules in the pre-training of large language models and finds that linear decay to zero (D2Z) achieves the lowest loss in both optimal computation and overfitting scenarios.
STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning
Marius Memmel (University of Washington), Jonathan Francis (Bosch Center for Artificial Intelligence)
RetrievalRobotic IntelligenceTransformerReinforcement LearningContrastive LearningSequential
🎯 What it does: Improved the performance of few-shot imitation learning by retrieving and training sub-trajectories at deployment.
Strategic Classification With Externalities
Safwan Hossain (Harvard University), Ariel D. Procaccia
ClassificationOptimizationTabular
🎯 What it does: This paper incorporates multi-agent externalities into the traditional strategic classification framework, proposing a Stackelberg-Nash equilibrium model and proving that the equilibrium is unique and can be efficiently solved under symmetric externalities and convex costs.
Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search
Jonathan Light (Rensselaer Polytechnic Institute), Ziniu Hu (California Institute of Technology)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A two-layer decision framework called STRATEGIST is implemented to achieve self-improvement in a self-play environment without training data by learning high-level strategies through large language models (LLM) and combining them with low-level Monte Carlo Tree Search (MCTS);
Streaming Algorithms For $\ell_p$ Flows and $\ell_p$ Regression
Amit Chakrabarti (Dartmouth), Taisuke Yasuda (Carnegie Mellon)
Optimization
🎯 What it does: This paper proposes a novel streaming algorithm for solving underdetermined ℓ_p regression (including ℓ_p flow problems) under the column-arrival stream model, which can obtain cost estimates or approximate solutions with only one scan and space much smaller than the number of input columns.
Streaming Video Question-Answering with In-context Video KV-Cache Retrieval
Shangzhe Di (Shanghai Jiao Tong University), Hao Jiang (Alibaba Group)
RetrievalComputational EfficiencyTransformerVision Language ModelVideo
🎯 What it does: We propose ReKV, a training-free framework that achieves efficient streaming video question answering through sliding window attention and context KV-Cache retrieval.
Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge
Haomiao Xiong (Dalian University of Technology), Huchuan Lu (Dalian University of Technology)
RecognitionRetrievalRecommendation SystemTransformerLarge Language ModelOptical FlowVideoMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: A framework for untrained streaming video understanding and multi-turn dialogue, STREAMCHAT, is proposed, along with a multi-type multi-turn interaction benchmark, STREAMBENCH.
Streamlining Prediction in Bayesian Deep Learning
Rui Li (Aalto University), Martin Trapp (Aalto University)
Computational EfficiencyTransformerSupervised Fine-TuningImageTabular
🎯 What it does: A single forward prediction method is proposed in Bayesian deep learning, achieved through local linearization and local Gaussian approximation, which can obtain an approximate posterior predictive distribution without sampling.
Streamlining Redundant Layers to Compress Large Language Models
Xiaodong Chen (Renmin University of China), Hong Chen (Renmin University of China)
ClassificationGenerationCompressionTransformerLarge Language ModelText
🎯 What it does: The LLM-Streamline method is proposed, which first performs hierarchical pruning on the LLM and then replaces the pruned layers with lightweight networks to reduce the number of parameters while maintaining performance.
Strength Estimation and Human-Like Strength Adjustment in Games
Chun Jung Chen, Ti-Rong Wu (Academia Sinica)
Reinforcement LearningTabular
🎯 What it does: A strength estimator (SE) based on human game records and a Monte Carlo Tree Search (SE-MCTS) based on this estimator are proposed to predict player levels in board games and adjust the AI's strength in a human-like manner.
StringLLM: Understanding the String Processing Capability of Large Language Models
Xilong Wang (Duke University), Neil Zhenqiang Gong (Duke University)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes the StringLLM method, constructs a large-scale string processing benchmark dataset called StringBench, and systematically evaluates the performance of multiple LLMs on string processing tasks.
Strong Model Collapse
Elvis Dohmatob (Meta), Julia Kempe (Meta)
Data SynthesisOptimizationTransformerLarge Language ModelImageText
🎯 What it does: This paper studies the phenomenon of 'Model Collapse' that occurs when the training set contains a mixture of real and synthetic data, and provides a theoretical proof and empirical validation of strong model collapse.
Strong Preferences Affect the Robustness of Preference Models and Value Alignment
Ziwei Xu (National University of Singapore), Mohan Kankanhalli (National University of Singapore)
Recommendation SystemTransformerLarge Language ModelReinforcement LearningTabular
🎯 What it does: This paper studies the sensitivity of the Bradley-Terry and Plackett-Luce preference models in the face of small probability changes through theoretical analysis and experimental validation, and explores the impact of this sensitivity on value alignment robustness.
StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization
Zhuoqun Li (Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences), Yongbin Li (Tongyi Lab)
TransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes the StructRAG framework, which can automatically select the most suitable structured form (table, graph, algorithm, directory, or block) in knowledge-intensive reasoning tasks and convert the original document into that structured knowledge. It then generates answers through question decomposition, knowledge extraction, and reasoning.
Structural-Entropy-Based Sample Selection for Efficient and Effective Learning
Tianchi Xie (Tsinghua University), Shixia Liu
ClassificationObject DetectionData-Centric LearningGraph Neural NetworkImageText
🎯 What it does: This paper proposes a sample selection method based on structural entropy (SES), aimed at efficiently selecting samples that are both informative and representative of the overall distribution in machine learning tasks.
Structure Language Models for Protein Conformation Generation
Jiarui Lu (Mila Quebec AI Institute), Jian Tang (Mila Quebec AI Institute)
GenerationProtein Structure PredictionTransformerDiffusion modelAuto EncoderBiomedical Data
🎯 What it does: A structural language model (SLM) framework is proposed, which quantizes protein three-dimensional conformations into latent corpora using discrete variational autoencoders. Subsequently, new conformations are generated in the latent space using conditional language models or masked diffusion, and are restored to three-dimensional space through a structural decoder.
Structuring Benchmark into Knowledge Graphs to Assist Large Language Models in Retrieving and Designing Models
Hanmo Liu (Hong Kong University of Science and Technology), Lei Chen (Hong Kong University of Science and Technology)
RetrievalNeural Architecture SearchGraph Neural NetworkLarge Language ModelGraphBenchmark
🎯 What it does: Construct a Knowledge Benchmark Graph (KBG) that structures data, models, and performance information into a graph, and uses this graph to retrieve the most suitable neural network models for unseen datasets.
Student-Informed Teacher Training
Nico Messikommer (University of Zurich), Davide Scaramuzza (University of Zurich)
Robotic IntelligenceReinforcement LearningTabular
🎯 What it does: This paper proposes a method for jointly training teacher and student strategies, aimed at addressing the imitation difficulties caused by the teacher having complete observations while the student has limited observations.
Studying the Interplay Between the Actor and Critic Representations in Reinforcement Learning
Samuel Garcin (University of Edinburgh), Stefano V Albrecht
Representation LearningReinforcement LearningVideo
🎯 What it does: This study investigates the complementarity and information specialization of the actor and critic in deep reinforcement learning when sharing and separating representations.
Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking
Benjamin Feuer (New York University), John P Dickerson
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper systematically evaluates the reliability of using LLM-judge for alignment benchmarks and finds that its implicit preferences tend towards style rather than factuality or safety. It proposes SOS-BENCH, a unified alignment evaluation consisting of 19 standard benchmarks, and demonstrates through large-scale experiments that data scale and prompt diversity are the main drivers for improving alignment during the post-training SFT phase, while a decline in world knowledge is observed during the PO phase.
Subgraph Federated Learning for Local Generalization
Sungwon Kim (KAIST), Chanyoung Park (KAIST)
Federated LearningSafty and PrivacyGraph Neural NetworkGraph
🎯 What it does: A subgraph federated learning framework named FedLoG is proposed, which utilizes reliable head node (head degree and head category) information to generate global synthetic data, and adapts missing knowledge during local training through a local generalization phase, thereby reducing local overfitting and enhancing generalization ability to unseen data.
Subtask-Aware Visual Reward Learning from Segmented Demonstrations
Changyeon Kim (KAIST), Kimin Lee (KAIST)
Robotic IntelligenceTransformerReinforcement LearningContrastive LearningVideo
🎯 What it does: Utilize segmented demonstration videos to learn visual rewards, generating a dense reward function perceived by sub-tasks and used for reinforcement learning.
Sufficient Context: A New Lens on Retrieval Augmented Generation Systems
Hailey Joren (University of California San Diego), Cyrus Rashtchian (Google)
GenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: This study investigates the concept of 'sufficient context' in retrieval-augmented generation (RAG) systems, constructs a self-evaluator to assess instances, analyzes model errors based on this, improves hallucinations, and proposes a selective generation method.
Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization
Wenkai Yang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
Reinforcement LearningTabular
🎯 What it does: This paper studies the security issue of strong model alignment under weak supervision—weak-to-strong deception—and quantifies and evaluates this phenomenon in a multi-objective alignment scenario.
SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction
Ling Yang (Peking University), Shuicheng YAN
OptimizationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: To address the difficulties in error localization and self-correction of small LLMs in complex mathematical reasoning, a two-stage SupERCORRECT framework is proposed, significantly improving the model's reasoning accuracy.
Supervised and Semi-Supervised Diffusion Maps with Label-Driven Diffusion
Harel Mendelman (Technion), Ronen Talmon (Technion)
ClassificationRepresentation LearningDiffusion modelMultimodalityTabular
🎯 What it does: Proposes Supervised Diffusion Maps (SDM) and Semi-Supervised Diffusion Maps (SSDM), which utilize label information to construct a multi-view adjacency matrix based on traditional Diffusion Maps, and achieve label-driven diffusion through multiplicative interpolation to obtain more task-relevant low-dimensional embeddings.
Support is All You Need for Certified VAE Training
Changming Xu (University of Illinois Urbana-Champaign), Gagandeep Singh (University of Illinois Urbana-Champaign)
GenerationAnomaly DetectionAuto EncoderImage
🎯 What it does: Proposes the CIVET method, which conducts provably robust training for Variational Autoencoders (VAE) by transforming the worst-case error into deterministic decoder error using a support set;
SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatars
Jaeseong Lee (Korea Advanced Institute of Science and Technology), Jaegul Choo (Korea Advanced Institute of Science and Technology)
RestorationGenerationData SynthesisGaussian SplattingImageVideo
🎯 What it does: This paper proposes the SurFhead method, which binds 2D Gaussian surfels with 3DMFM to reconstruct geometrically accurate and controllable head avatars from RGB videos.
Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation
Xinpeng Wang (Ludwig Maximilian University of Munich), Barbara Plank (Bocconi University)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes a surgical method to reduce the false refusal rate of language models by truncating a single vector (i.e., the false refusal vector) in the Transformer activation stream.
Surprising Effectiveness of pretraining Ternary Language Model at Scale
Ayush Kaushal (Nolano AI), Irina Rish (Nolano AI)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This study investigates low-bit-width language models, proposing and training a ternary language model (TriLM), and systematically comparing it with traditional floating-point models (FloatLM) and post-training quantized models (QuantLM) at different scales and bit widths, while constructing the publicly available Spectra LLM suite.
SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding
Jian Chen (University at Buffalo), Tong Sun (Adobe Research)
RetrievalTransformerLarge Language ModelSupervised Fine-TuningTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposes the SV-RAG framework, utilizing a multimodal large language model (MLLM) to achieve long document retrieval and question answering.
SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency
Yiming Xie (Stability AI), Varun Jampani (Stability AI)
GenerationData SynthesisOptimizationDiffusion modelNeural Radiance FieldVideo
🎯 What it does: A unified video diffusion model SV4D is proposed, capable of generating multi-frame, multi-view consistent dynamic 3D (4D) content from a single-view video, and directly utilizing the generated view videos for efficient optimization of the 4D representation without the time-consuming SDS loss.
SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding
Zhenyu Yang (Kuaishou Technology), Changsheng Xu (Peng Cheng Laboratory)
RecognitionRetrievalRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextBenchmark
🎯 What it does: A long-term streaming media video multi-turn dialogue benchmark named SVBench has been constructed, releasing 49,979 QA pairs and 1,353 videos, and further training the StreamingChat model to enhance streaming video understanding performance.
SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression
Xin Wang (Ohio State University), Mi Zhang (Michigan State University)
GenerationCompressionTransformerLarge Language ModelText
🎯 What it does: A method called SVD-LLM is proposed for post-training compression of large language models (LLMs) without retraining.
SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models
Muyang Li (Massachusetts Institute of Technology), Song Han (Massachusetts Institute of Technology)
GenerationCompressionDiffusion modelImageText
🎯 What it does: This paper proposes SVDQuant, a 4-bit post-training quantization method that achieves low-bit inference for high-quality Diffusion models by absorbing outliers in weights and activations through low-rank branches.
SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix
Peng Dai (Google), Yinda Zhang (The University of Hong Kong)
GenerationData SynthesisDepth EstimationDiffusion modelOptical FlowVideo
🎯 What it does: A posture-free and training-free framework is proposed, utilizing a pre-trained monocular video generation model and depth estimation to convert monocular videos into stereo videos.
SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?
John Yang (Stanford University), Ofir Press (Meta AI)
AI Code AssistantTransformerLarge Language ModelImageVideoTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: A new benchmark called SWE-bench Multimodal (SWEbench M) was constructed and evaluated to test the performance of code generation models on front-end JavaScript software engineering tasks that include images and visual elements.
SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement
Antonis Antoniades (University of California), William Yang Wang (National University of Singapore)
OptimizationAI Code AssistantLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: This paper presents SWE-Search, a multi-agent framework that combines MCTS and self-feedback to improve software engineering tasks.
SWEb: A Large Web Dataset for the Scandinavian Languages
Tobias Norlund (AI Sweden), Magnus Sahlgren (AI Sweden)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: SWEb is proposed and released, a pre-trained dataset covering Swedish, Danish, Norwegian, and Icelandic with a scale of 1 trillion tokens, along with a model-based Markdown text extractor.
Swift Hydra: Self-Reinforcing Generative Framework for Anomaly Detection with Multiple Mamba Models
Nguyen Hoang Khoi Do (University of Florida), My T. Thai (University of Florida)
Anomaly DetectionReinforcement LearningMixture of ExpertsAuto EncoderTabularBenchmark
🎯 What it does: The Swift Hydra framework is proposed, which uses reinforcement learning to guide C-VAE in generating diverse and hard-to-detect synthetic anomaly samples, and then trains an efficient anomaly detector using the Mamba model with a MoE structure;
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
Heming Xia (Hong Kong Polytechnic University), Wenjie Li
GenerationOptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A self-reflective reasoning acceleration method called SWIFT is proposed, which utilizes the hierarchical sparsity of LLM to dynamically skip intermediate layers during inference, forming a lightweight draft model for Draft-Verify.
Swift4D: Adaptive divide-and-conquer Gaussian Splatting for compact and efficient reconstruction of dynamic scene
Jiahao Wu (Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology), Ronggang Wang (Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology)
RestorationCompressionComputational EfficiencyNeural Radiance FieldGaussian SplattingPoint Cloud
🎯 What it does: Developed Swift4D, which achieves fast and high-quality 4D scene reconstruction by separating dynamic and static points, using 4D hash time mapping only for dynamic points, and applying time importance pruning.
Swing-by Dynamics in Concept Learning and Compositional Generalization
Yongyi Yang (Harvard University), Hidenori Tanaka (Harvard University)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: The Structured Identity Mapping (SIM) task is proposed as a theoretical abstraction of the concept space, and it is used to study the learning dynamics of neural networks in combinatorial generalization; by analyzing the gradient flow of linear and symmetric two-layer linear models, the so-called Swing-by dynamics and the resulting multiple descent phenomenon are revealed, and these predictions are validated on text-conditioned diffusion models.
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning
Yuxiang Lu (Shanghai Jiao Tong University), Yu-Xiong Wang (University of Illinois Urbana-Champaign)
SegmentationKnowledge DistillationTransformerMixture of ExpertsImage
🎯 What it does: This paper proposes a multi-teacher knowledge distillation framework called Swiss Army Knife (SAK), which effectively integrates the different representation biases of multiple visual foundation models (VFM) through a teacher-independent backbone, teacher-specific adapter paths, and a mixture representation router to enhance multi-task learning performance.
Sylber: Syllabic Embedding Representation of Speech from Raw Audio
Cheol Jun Cho (University of California), Gopala Anumanchipalli
CompressionKnowledge DistillationRepresentation LearningTransformerAudio
🎯 What it does: A self-supervised learning framework called Sylber is proposed, which maps raw audio to clearly structured syllable-level embeddings, achieving linear-time syllable segmentation and low-bitrate reconstruction.
SyllableLM: Learning Coarse Semantic Units for Speech Language Models
Alan Baade (University of Texas at Austin), David Harwath (University of Texas at Austin)
RecognitionComputational EfficiencyKnowledge DistillationTransformerSupervised Fine-TuningAudio
🎯 What it does: This paper proposes an unsupervised method that extracts coarse semantic units of speech (similar to syllables) by analyzing the loss distribution of pre-trained self-supervised models, and refines the feature space using iterative distillation (SylBoost), ultimately generating low-bitrate discrete labels for training the Speech Language Model (SyllableLM), achieving significant improvements in training and inference speed.
Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length
Zihan Yu (Tsinghua University), Depeng Jin (Tsinghua University)
TransformerTabularBenchmark
🎯 What it does: This paper proposes a symbol regression method based on Minimum Description Length (MDL) called SR4MDL. By training the MDLformer network to predict the MDL of the data and using Monte Carlo Tree Search (MCTS) to prioritize exploring paths that reduce MDL in the search space, the formula recovery rate is significantly improved.
SymDiff: Equivariant Diffusion via Stochastic Symmetrisation
Leo Zhang (University of Oxford), Rob Cornish (University of Oxford)
GenerationDrug DiscoveryTransformerDiffusion modelGraph
🎯 What it does: This paper proposes SYMDIFF, a method for constructing equivariant diffusion models based on random symmetrization, and applies it to E(3) equivariant molecular generation.
SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models
Daniel Levy (McGill University), Siamak Ravanbakhsh (Intel Labs)
GenerationData SynthesisComputational EfficiencyDiffusion modelGraphPhysics Related
🎯 What it does: A diffusion model is proposed that utilizes asymmetric units and point group symmetry information to achieve precise generation of crystal space group symmetries.
SymmetricDiffusers: Learning Discrete Diffusion on Finite Symmetric Groups
Yongxing Zhang (University of Waterloo), Renjie Liao (University of British Columbia)
OptimizationTransformerDiffusion modelImage
🎯 What it does: A discrete diffusion model called SymmetricDiffusers is proposed on the finite symmetric group $S_n$ to learn permutation distributions and solve sorting, jigsaw, and traveling salesman problems.
SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints
Jianhong Bai (Zhejiang University), Di ZHANG
GenerationData SynthesisTransformerDiffusion modelRectified FlowImageVideo
🎯 What it does: A multi-camera synchronized video generation framework called SynCamMaster is proposed, which can generate consistent videos from any viewpoint.
Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling
Cristian Rodriguez-Opazo (Australian Institute for Machine Learning), Anton van den Hengel (Australian Institute for Machine Learning)
ClassificationTransformerContrastive LearningImage
🎯 What it does: This study investigates the differences in zero-shot classification performance of CLIP visual encoders (such as ResNet and ViT) across different datasets and proposes a neural logit controller (NLC) based on adaptive temperature scaling to integrate multiple backbones to improve classification accuracy.
Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learning
Zijian Li (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
GenerationRepresentation LearningAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: A recognizable decoupled representation learning framework is proposed, combining sufficient variation and sparse mixture assumptions, and implementing two models: CG-VAE and CG-GAN.