International Conference on Machine Learning Β· 1032 papers
SE-GA: Memory-Augmented Self-Evolution for GUI Agents
Shilong Jin (Tianjin University), Zhuosheng Zhang (Shanghai Jiao Tong University)
CodeAutonomous DrivingOptimizationRobotic IntelligenceTransformerReinforcement LearningAgentic AIVision Language ModelImageTextMultimodalityRetrieval-Augmented Generation
π― What it does: Propose the SE-GA framework, integrating Test-Time Memory Expansion (TTME) with Self-Evolving Training (MASE), enabling GUI agents to dynamically retrieve historical episodes, semantic, and experiential memories during multi-step tasks, and continuously learn and optimize through online interaction;
Felix Laarmann (TU Dortmund University), Jakob Rehof (TU Dortmund University)
CodeOptimizationHyperparameter SearchData-Centric LearningNeural Architecture SearchReinforcement LearningContrastive LearningGaussian SplattingTabularTime Series
π― What it does: This paper proposes a framework based on Finite Combinatory Logic with Parameters (FCLP) and Para-construction, used to automatically synthesize the search space of parameterized functions and perform search within this space.
Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs
Wei-Yao Wang (Sony Group Corporation), Yoshiyuki Kobayashi (Sony Group Corporation)
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
π― What it does: Proposed a Modal Mutual Attention (MMA), which unlocks the causal attention in the decoder, allowing image tokens to attend to text tokens, thereby alleviating the vision-language mismatch and target misreporting issues in Multimodal Large Language Models (MLLMs).
π― What it does: This paper proposes an efficient video transfer framework that converts VLA videos from simulated environments into realistic videos to enhance training data while preserving task semantics and action trajectories.
π― What it does: Propose a new framework called SegPVSG for Panoptic Video Scene Graph Generation (PVSG), which includes two modules: Temporal Focus Network (TFN) and Relation-Centric Generative Video Augmentation (RGVA);
π― What it does: Propose a data-agnostic quantization method called MaskAQ, which utilizes information regions in self-attention to perform masked attention alignment, thereby generating high-quality synthetic samples for low-bit quantization of Vision Transformers.
π― What it does: A new self-supervised pre-training framework called PULSE is studied, which utilizes dynamic system models and cross-reconstruction tasks to extract transferable system information and suppress sample-specific noise, specifically for physiological time series.
SEMA: a Scalable and Efficient Mamba like Attention via Token Localization and Averaging
Nhat Thanh Tran (University of California, Irvine), Jack Xin (University of California, Irvine)
CodeClassificationObject DetectionSegmentationComputational EfficiencyTransformerMixture of ExpertsContrastive LearningImageVideo
π― What it does: Propose a scalable and efficient Mamba-style attention mechanism called SEMA, which utilizes window localization to avoid attention dispersion and achieves global information fusion through arithmetic averaging (mean mixing), addressing the problem of attention focus loss in long sequences, and verifying its effectiveness in various visual tasks.
Semantic-Aware Motion Encoding for Topology-Agnostic Character Animation
Zongye Zhang (Beihang University), Yunhong Wang (Beihang University)
CodeGenerationData SynthesisPose EstimationGraph Neural NetworkTransformerLarge Language ModelVision-Language-Action ModelAuto EncoderContrastive LearningVideoTextMultimodality
π― What it does: Proposes a semantic-aware, topology-agnostic motion encoding framework called SATA, which can map actions from any skeletal structure into a unified latent space and enable reproduction and cross-species retargeting.
Semi-knockoffs: a model-agnostic conditional independence testing method with finite-sample guarantees
Angel David REYERO LOBO (University of Toulouse), Pierre Neuvial (University of Paris-Saclay)
CodeOptimizationFederated LearningExplainability and InterpretabilityData-Centric LearningContrastive LearningTabularBiomedical Data
π― What it does: This paper proposes a model-agnostic conditional independence testing method called Semi-knockoffs, which can directly use any pre-trained machine learning model without requiring training-test splits; it provides finite sample type-I error and FDR control in high-dimensional settings; and provides theoretical support, including the optimization stability and double robustness of regularized learners with irrelevant features.
π― What it does: Propose a semi-supervised learning framework called DSCL, which achieves gaze estimation by utilizing a small amount of labeled data and a large amount of unlabeled data. It mainly achieves feature separation through Jacobian regularization, and then improves generalization performance by performing contrastive learning and sequence ranking in each subspace.
Semi-Supervised Learning for Molecular Graphs via Ensemble Consensus
Rasmus Hannibal Tirsgaard (Technical University of Denmark), Mikkel N. Schmidt (Technical University of Denmark)
CodeDrug DiscoveryGraph Neural NetworkMixture of ExpertsContrastive LearningGraph
π― What it does: Propose a semi-supervised learning framework based on integrated consensus, leveraging unlabeled molecular graph data to improve prediction performance.
π― What it does: Propose a sequential conditional independence test method based on the betting framework and self-normalized kernel conditional independence statistic.
π― What it does: Propose a method for generating synthetic trajectories through Soft-Verified Generation (SVG), utilizing line-level recall for soft verification to eliminate the dependency on unit tests, thereby enabling fast and low-cost training of code agents for any codebase.
Set-Preserving Calibration from Conformal P-Values to E-Values
Nabil Alami (Mohamed bin Zayed University of Artificial Intelligence), Souhaib Ben Taieb (Mohamed bin Zayed University of Artificial Intelligence)
CodeFederated LearningExplainability and InterpretabilityComputational EfficiencyData-Centric LearningTabularBenchmark
π― What it does: Proposed a Set-Preserving calibration method from conformal p-value to e-value (P2E), and applied it to cross-conformal prediction (CCP) and conformal aggregation (CA).
π― What it does: This paper proposes a sequential visual localization method based on 3D scene graphs and particle filters, utilizing semantic features and coarse grids to achieve efficient localization.
SGERA: Stein-Guided ECG-Report Alignment for ECG Representation Learning
Jian Chen (University of Hong Kong), Edith Cheuk-Han Ngai (University of Hong Kong)
CodeDomain AdaptationRepresentation LearningConvolutional Neural NetworkTransformerScore-based ModelContrastive LearningTextMultimodalityElectronic Health RecordsElectrocardiogram
π― What it does: Proposed the SGERA framework, which uses the Stein kernel to achieve dual-level alignment between ECG and reports, addressing the limitations of traditional CLIP-style alignment in cross-modal distribution differences;
ShapCCS: Shapley-Driven Client Coreset Selection in Federated Learning
Shuo Ji (Southwest Jiaotong University), Jie Xu (University of Leeds)
CodeFederated LearningExplainability and InterpretabilityComputational EfficiencyContrastive LearningImage
π― What it does: Proposes ShapCCS, a client core sample selection method based on Shapley values, aimed at significantly reducing computational and communication costs in federated learning.
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningRecurrent Neural NetworkTransformerDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningTabularTime SeriesBiomedical DataBenchmarkPhysics Related
π― What it does: Proposed an information theory-based global feature importance measure (Info-Shap) and corresponding differentiable regularizations (Shap and F-Shap), embedding them into neural network training to improve neural Granger causal discovery.
Shared Semantics, Divergent Mechanisms: Unsupervised Feature Discovery by Aligning Semantics and Mechanisms
Hyunjin Cho (Yonsei University), Jaehyung Kim (Yonsei University)
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelContrastive LearningText
π― What it does: Propose a distribution-level unsupervised feature discovery method that combines semantic embeddings and sequence-level mechanism attribution to cluster the diverse continuations generated by LLMs;
Sharp Empirical Bernstein Inequalities for the Variance of Bounded Random Variables
Diego Martinez-Taboada (Carnegie Mellon University), Aaditya Ramdas (Carnegie Mellon University)
CodeOptimizationExplainability and InterpretabilityComputational EfficiencyTabularTime SeriesReview/Survey Paper
π― What it does: This paper proposes a fully empirical Bessel inequality for the variance of bounded random variables, providing confidence intervals and confidence sequences for both batch and sequential settings.
SHERPA: Fine-tuning Segment Anything Models with Task-relevant Guidance
Jingcheng Xie (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
CodeSegmentationKnowledge DistillationRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningImageVideoBiomedical Data
π― What it does: Propose the SHERPA framework, which utilizes task-related features from a small SAM to guide the fine-tuning of a large SAM, thereby enhancing performance on specialized tasks while maintaining its generality.
Shortcut-Resistant CAM Distillation for Long-Tailed Recognition
Wenhai Wan (National Engineering Research Center for Big Data Technology and System), Songcan Chen (MIIT Key Laboratory of Pattern Analysis and Machine Intelligence)
π― What it does: This paper proposes a framework called Shortcut-Resistant CAM Distillation (SRCD), which utilizes Class Activation Maps (CAM) to transfer the object-centric attention learned from the head classes with large sample sizes to the tail classes with scarce samples, thereby suppressing the model's reliance on spurious features in long-tailed scenarios.
Signature-Informed Transformer for Asset Allocation
Yoontae Hwang (Pusan National University), Stefan Zohren (University of Oxford)
CodeOptimizationTransformerTime SeriesFinance Related
π― What it does: Propose a unified end-to-end asset allocation model (Signature-Informed Transformer, SIT) that combines path signatures with Transformer, which performs feature extraction and directly outputs portfolio weights, avoiding error amplification in the traditional prediction-then-optimization process;
π― What it does: Propose a graph generation method called SimGFM based on Discrete Flow Matching (DFM), achieving efficient sampling through a minimalist design.
Simple Policy Gradients for Reasoning with Diffusion Language Models
Anthony Zhan (Stanford University)
CodeTransformerLarge Language ModelReinforcement LearningPrompt EngineeringDiffusion modelText
π― What it does: The AGRPO algorithm is proposed for discrete diffusion language models, utilizing a step-level Markov decision process to achieve post-training, significantly improving inference consistency and quality.
SKETCH: Semantic Key-Point Conditioning for Long-Horizon Vessel Trajectory Prediction
Linyong Gan (Chinese University of Hong Kong), Shuhang Chen (COSCO SHIPPING Advanced Technology Institute)
CodeAutonomous DrivingOptimizationComputational EfficiencyRepresentation LearningReinforcement Learning from Human FeedbackTransformerDiffusion modelAuto EncoderContrastive LearningTime SeriesSequentialRetrieval-Augmented Generation
π― What it does: Propose a hierarchical ship trajectory prediction framework based on semantic key points (NKP), decomposing long-term trajectory prediction into two steps: global intent and local motion.
Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills
Justin Chen, Mohit Bansal (University Of North Carolina Chapel Hill)
CodeExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationRepresentation LearningMeta LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelAgentic AIPrompt EngineeringMixture of ExpertsTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Propose SKILL-MOE, a Mixture-of-Experts framework that utilizes existing pre-trained large language models during the inference phase. It achieves instance-level expert selection by inferring the discrete skills required for each question, and synthesizes the final answer by aggregating the Chain-of-Thought results generated by each expert.
π― What it does: Designed the Skill-Pro framework, which automatically learns and reuses executable procedural skills (Skill) in LLM agents using non-parametric PPO.
SkillNet: Hierarchical Skill Modeling for Compositional Generalization in Vision-Language Action Models
Senwei Xie (Chinese Academy of Sciences), Xilin CHEN
CodeRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerMixture of ExpertsVision-Language-Action ModelFlow-based ModelContrastive LearningImageVideoTextMultimodality
π― What it does: Propose SkillNet, a visual-language-action model that achieves compositional generalization through hierarchical skill modeling (mechanical attributes and semantic attributes) combined with a skill-contextualized Mixture-of-Experts (SCMoE).
Skip a Layer or Loop It? Learning Program-of-Layers in LLMs
Ziyue Li (University of Maryland), Tianyi Zhou (MBZUAI)
CodeComputational EfficiencyAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
π― What it does: The study dynamically skips or loops layers during LLM inference to construct input-specific program execution.
SLQ: Bridging Modalities via Shared Latent Queries for Retrieval with Frozen MLLMs
Haoran Lou (Beijing University of Posts and Telecommunications), Xu Tang (Xiaohongshu Inc)
CodeRetrievalDomain AdaptationRepresentation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
π― What it does: Propose the SLQ framework, which efficiently transfers the model into a retriever by adding a small number of shared latent queries (Shared Latent Queries) behind a frozen multimodal large language model, preserving pre-trained knowledge and inference capabilities.
SMART: Scalable Meshβfree Aerodynamic Simulations from Raw Geometries using a Transformerβbased Surrogate Model
Jan Hagnberger (University of Stuttgart), Mathias Niepert (University of Stuttgart)
CodeOptimizationComputational EfficiencyTransformerAuto EncoderContrastive LearningPoint CloudMeshPhysics Related
π― What it does: Proposes SMART, a grid-free aerodynamic simulation surrogate model based on Transformer, capable of predicting physical quantities using only point cloud geometry and query points, completely independent of simulation grids.
CodeComputational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextChain-of-Thought
π― What it does: Propose SmartThinker, an efficient inference method based on GRPO, which compresses the reasoning trajectory of large language models by dynamically estimating the optimal length of chain-of-thought and introducing dynamic length rewards.
π― What it does: Propose SmoothSA, which addresses the cold start and homogeneity issues of frame-to-frame transformation in the first frame of images/videos by introducing preheated queries with different iteration counts across frames, thereby improving the quality of object-centric learning.
π― What it does: Research and implement SoftMatcha 2, a high-speed soft pattern matching algorithm for tera-scale corpora, supporting semantic variations such as word substitution, insertion, and deletion;
SoftMoE: Soft Differentiable Routing for Mixture-of-Experts in LLMs
MikoΕaj Zasada (AGH University of Krakow), Marcin Kurdziel (AGH University of Krakow)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelMixture of ExpertsContrastive LearningText
π― What it does: Propose SoftMoE, a sparse Mixture-of-Experts routing mechanism that uses LapSum soft topk approximation; achieve adaptive allocation of expert capacity through differentiable soft routing; and reduce the number of activated experts while maintaining autoregressive compatibility.
Solving the Offline and Online Min-Max Problem of Non-smooth Submodular-Concave Functions: A Zeroth-Order Approach
Amir Ali Farzin (Australian National University), Iman Shames (University of Melbourne)
CodeOptimizationImage
π― What it does: This paper studies the max-min and min-max problems involving submodular functions that may be non-smooth with respect to the minimizer and concave functions with respect to the maximizer, and proposes a solution based on zeroth-order methods.
π― What it does: Proposes SonicMaster, a unified text-driven generative model for simultaneously repairing 19 common distortions in music (such as reverb, distortion, attenuation, dynamic compression, stereo imbalance, etc.), and supports users to achieve controllable repair through natural language instructions or automatic global repair;
SorryDB: Can AI Provers Complete Real-World Lean Theorems?
Austin Letson (Axiomatic AI), Lenny Taelman (University of Amsterdam)
CodeAI Code AssistantTransformerLarge Language ModelAgentic AIPrompt EngineeringTextBenchmark
π― What it does: Proposes a dynamically updated Lean task benchmark SorryDB, aiming to evaluate the practicality of AI provers in real formalization projects.
SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport
Simon Roschmann (Helmholtz Munich), Zeynep Akata (Helmholtz Munich)
CodeRetrievalDomain AdaptationRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: We construct a semi-supervised alignment framework called SOTAlign on pre-trained unimodal vision and language encoders, using only a small number of image-text paired samples and a large amount of unpaired image/text data. It first performs rough alignment via a linear teacher, and then refines cross-modal embeddings using KLOT divergence based on Optimal Transport.
CodeData SynthesisExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
π― What it does: By utilizing seven manually designed prompt templates based on principles of cognitive learning, LLMs are repeatedly used to rewrite a small-scale professional corpus, generating a large-scale synthetic corpus. Continued pre-training on this synthetic corpus enables knowledge injection.
SpaEF: Spatially Resolved Transcriptomics Data Element-Wise Denoising Framework Powered by Large Models
Zekuan Shang (Jilin University), You Zhou (Jilin University)
CodeRestorationGraph Neural NetworkTransformerLarge Language ModelAuto EncoderContrastive LearningGraphBiomedical Data
π― What it does: Propose and implement SpaEF, a framework that constructs spot and gene graphs using large models and achieves denoising of spatial transcriptomics data through element-wise graph autoencoders.
SPARe: Stacked Parallelism with Adaptive Reordering for Fault-Tolerant LLM Pretraining Systems with 100k+ GPUs
Jin Lee (University of California, Santa Barbara), Zheng Zhang (University of California, Santa Barbara)
CodeOptimizationFederated LearningComputational EfficiencyTransformerLarge Language ModelText
π― What it does: Propose the SPARe framework, which achieves fault tolerance in large LLM pretraining by utilizing stacked parallelism and adaptive reordering, avoiding frequent global restarts.
Sparse and Faithful Local Explanations with Piecewise Linear Surrogates
Yixin Wang (Sichuan University), Yucheng Dong (Sichuan University)
CodeExplainability and InterpretabilityTabular
π― What it does: Proposes the PL-LIME framework, which uses instance-anchored piecewise linear proxies to provide local interpretability for black-box models;
Sparse Bayesian Deep Functional Learning with Structured Region Selection
Xiaoxian Zhu (Shanghai University of Finance and Economics), Mengyun Wu (Shanghai University of Finance and Economics)
CodeExplainability and InterpretabilityComputational EfficiencyData-Centric LearningDiffusion modelScore-based ModelContrastive LearningTabularTime SeriesElectrocardiogramAudio
π― What it does: Proposed a sparse Bayesian deep functional neural network (sBayFDNN) for nonlinear scalar-to-function regression and automatic identification of functional subregions.
Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
David Chanin (University College London), AdriΓ Garriga-Alonso (MATSResearch)
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelAuto EncoderContrastive LearningTextSequential
π― What it does: Investigates how improper L0 settings (the average number of activated latent dimensions per input) in sparse autoencoders (SAE) can lead to feature mixing, thereby compromising uniqueness, and proposes a proxy metric based on the decoder's diagonal cosine similarity (c_dec) to determine the correct L0.
Sparse Regression with $\ell_0$ Constraints for $\alpha$-Mixing Time Series: Algorithms and Guarantees
Ruoxin Yuan (Fudan University), Lijun Ding (University of California San Diego)
CodeOptimizationExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTime Series
π― What it does: In time series data, the β0-constrained least squares problem for sparse linear regression on Ξ±-mixing Gaussian processes is solved, and high probability RSC/RSS properties are provided along with sample and iteration complexity analysis for exact sparse algorithms such as IHT, CoSaMP, and Subspace Pursuit; subsequently, the theory is applied to sparse VAR models, and the method is validated on synthetic VAR data and real New York City ride-sharing time series.
π― What it does: Proposed an scalable sparse communication mechanism called SOPS, which achieves dynamic sparse connections in large-scale multi-agent reinforcement learning through an exponential graph backbone and learnable subgraphs.
SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding
Shenggui Li (Nanyang Technological University), Tianwei Zhang (Nanyang Technological University)
CodeComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringMixture of ExpertsContrastive LearningTextRetrieval-Augmented Generation
π― What it does: This paper proposes the SpecForge framework and the SpecBundle toolkit, aiming to efficiently train and deploy speculative decoding draft models.
SpecPL: Disentangling Spectral Granularity for Prompt Learning
Jingtao Zhou (City University of Hong Kong), Lai Man Po
CodeClassificationRecognitionDomain AdaptationRepresentation LearningAdversarial AttackTransformerPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderContrastive LearningImageTextMultimodalityBenchmark
π― What it does: By decomposing low-frequency semantics and high-frequency details in the frozen VAE latent space, and introducing adversarial detail supervision during training, improving VLM prompt learning to eliminate modality asymmetry;
π― What it does: Propose Speculative Coupled Decoding (SCD), a training-free, lossless acceleration method for autoregressive visual generation models.
π― What it does: This paper proposes Langevin Speculative Dynamics (LSD), a distributed explicit sampling method that utilizes a fast draft model to generate steps and parallel verification at the backend, specifically designed to accelerate molecular dynamics simulations based on machine learning potentials.
π― What it does: Propose SpeedCP, an efficient and adjustable method for constructing prediction intervals through kernel RKHS conditional conformal prediction.
Spherical Steering: Geometry-Aware Activation Rotation for Language Models
Zejia You (Tufts University), Hanjie Chen (Rice University)
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelContrastive LearningText
π― What it does: Propose a geometry-based method for activation rotation during inference called Spherical Steering, which utilizes contrastive samples to construct directional prototypes and achieves control over the behavior of language models by performing norm-preserving rotations on the spherical surface in the hidden layer;
π― What it does: By analyzing the energy migration phenomenon in the time-frequency domain when the spiking camera focuses, an automatic focusing method called CEN based on the frequency domain spectral centroid is proposed.
π― What it does: A pulse Transformer named Spike-HTR is designed, which achieves efficient hand-written text line recognition by utilizing two budgets: short time slot numbers and width sequence length.
SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks
Srivatsa R Kundurthy, John Ling (Longitude Labs Inc.)
CodeGenerationData SynthesisRecommendation SystemExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTabularBenchmarkRetrieval-Augmented Generation
π― What it does: Proposed the SPREADSHEETARENA platform to evaluate the performance of LLMs in generating complete spreadsheet workbooks, and made the benchmark data and preference voting publicly available.
SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
Zhenyi Shen (King's College London), Xing Sun (Tencent)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelContrastive LearningTextBenchmark
π― What it does: Studied a training framework named SSA, which integrates sparse attention with full attention, and enhances performance under both sparse and full inference modes through bidirectional attention output alignment.
Paul Janson (Concordia University), Eugene Belilovsky (Concordia University)
CodeOptimizationComputational EfficiencyRepresentation LearningTransformerLarge Language ModelContrastive LearningText
π― What it does: Train large language models from scratch using low-rank decomposition, completely eliminating full-rank auxiliary weights, achieving end-to-end low-rank pre-training;
π― What it does: Propose SPPO, which introduces three latent-space regularization methods (CKA alignment, no-flip constraint, KDE exploration shaping) on top of PPO to improve training stability and performance in continuous control tasks.
Stable Localized Conformal Prediction via Transduction
Yinjie Min (Nankai University), Changliang Zou (University of Melbourne)
CodeClassificationDomain AdaptationAnomaly DetectionTabularBiomedical Data
π― What it does: Proposed a stabilization method called Stable Conformal Prediction (StCP), which utilizes source task labels and unlabelled target data to stabilize the size of prediction sets through transfer learning;
Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance
Minchan Kwon (KAIST), Junmo Kim (KAIST)
CodeAdversarial AttackReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningFlow-based ModelGenerative Adversarial NetworkContrastive LearningText
π― What it does: This paper proposes the Stable-GFN framework, improving Generative Flow Networks (GFN) for red team attacks on large language models (LLM), addressing the instability of the partition function Z estimation and the mode collapse caused by noisy rewards in traditional GFN.
π― What it does: Propose a two-stage strategy based on a single diffusion model (MAP estimation + re-noised posterior sampling), achieving traversal of the distortion-perception trade-off in zero-shot inverse problems.
STAND: Self-Aware Precondition Induction for Interactive Task Learning
Daniel Weitekamp (Georgia Institute of Technology), Christopher J. MacLellan (Georgia Institute of Technology)
CodeClassificationOptimizationComputational EfficiencyRepresentation LearningMeta LearningReinforcement Learning from Human FeedbackTransformerReinforcement LearningPrompt EngineeringContrastive LearningTabular
π― What it does: Proposed the STAND method for small-sample conditional pre-induction in interactive task learning, with the ability to self-assess learning progress.
Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Control
Ali Taghibakhshi (Nvidia Corporation), Pavlo Molchanov (Nvidia Corporation)
CodeComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelMixture of ExpertsText
π― What it does: Propose a post-training method called Star Elastic, which generates multiple-sized nested sub-models through a single training process and achieves elastic budget control during the inference phase.
STAR: Rethinking MoE Routing as Structure-Aware Subspace Learning
Sumin Park (Korea Advanced Institute of Science and Technology), Noseong Park (Korea Advanced Institute of Science and Technology)
CodeOptimizationFederated LearningComputational EfficiencyRepresentation LearningData-Centric LearningLarge Language ModelSupervised Fine-TuningMixture of ExpertsContrastive LearningImageTextMultimodalityTabularBenchmark
π― What it does: A STAR (Structure-Aware Routing) framework is proposed within Mixture-of-Experts (MoE) models, which utilizes online Principal Subspace learning (Generalized Hebbian Algorithm, GHA) to model the input structure, and combines it with traditional linear gating to achieve more stable and specialized expert routing.
Steer Like the LLM: Activation Steering that Mimics Prompting
Geert Heyman (Nokia Bell Labs), Frederik Vandeputte (Nokia Bell Labs)
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: The study proposes a framework that treats prompt steering as an activation-based guidance approach, and approximates the effect of prompts by learning token-level activation coefficients;
Steering Large Language Models through the DMTA Cycle: Structure-Based Drug Design via Knowledge-Driven Bi-Level Thompson Sampling
Xuanning Hu (Jilin University), Bo Yang (Jilin University)
CodeDrug DiscoveryTransformerLarge Language ModelReinforcement LearningDiffusion modelBiomedical DataRetrieval-Augmented Generation
π― What it does: By constructing a K-BTS framework based on dual-layer Thompson sampling, utilizing LLM to achieve closed-loop iterative optimization for structured drug design in DMTA
π― What it does: Propose a lightweight warm-start mechanism (spatiotemporal consistency prediction) and velocity-aware perturbation injection to accelerate diffusion-based audio-visual motion control strategies, enabling high-quality action generation in just 2 steps.
Stochastic Sparse Attention for Memory-Bound Inference
Kyle Lee (University of California Santa Barbara), Kerem Yunus Camsari
CodeComputational EfficiencyTransformerLarge Language ModelText
π― What it does: In long-context autoregressive decoding, SANTA (Stochastic Additive No-mulT Attention) is introduced, which significantly reduces the KV cache read bandwidth by sampling a very small number of value rows from the softmax distribution and aggregating them. Additionally, SΒ²ANTA provides hierarchical/systematic sampling variants to reduce variance, and implements GPU-friendly sampling and addition kernels. Furthermore, Bernoulli qKα΅ sampling is proposed to achieve sparse feature access during the score phase.
Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding
Yanzheng Xiang (Kings College London), Yulan He (Kings College London)
CodeComputational EfficiencyAI Code AssistantTransformerLarge Language ModelPrompt EngineeringDiffusion modelTextBenchmark
π― What it does: Propose the COVER method to solve the flip-flop oscillation in revocable diffusion decoding, improving the efficiency of parallel decoding.
STORM: Segment, Track, and Object Re-Localization from a Single Image
Yu Deng (Technical University of Darmstadt), Kristian Kersting (Technical University of Darmstadt)
CodeObject TrackingSegmentationPose EstimationTransformerVision Language ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageVideo
π― What it does: Propose the STORM framework, which achieves automated 6D pose estimation and tracking with only a single reference image, and can automatically relocalize when the target is missing;
Chris Hays (Massachusetts Institute of Technology), Manish Raghavan
CodeRecommendation SystemLarge Language ModelScore-based ModelContrastive LearningTextBenchmark
π― What it does: This paper investigates the issue of ranking manipulation caused by model cloning in generative AI arenas, proving that the traditional Bradley-Terry maximum likelihood ranking mechanism is not robust against cloning, and proposes a You-Rank-We-Rank (YRWR) mechanism that utilizes producer self-evaluation rankings to achieve approximate cloning robustness.
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextBenchmark
π― What it does: Propose the SAGE framework, which improves automated optimization modeling through explicit modeling strategies combined with reinforcement learning
π― What it does: Propose the StructMamPose framework, combining the Identity Anchoring Mechanism (IAM) and the View Transformation Hub (VTH) to improve monocular 3D human pose estimation.
CodeTabularTime SeriesBiomedical DataElectronic Health RecordsBenchmark
π― What it does: To address the problem of subgroup discovery in the Cox model for survival analysis, two new evaluation metrics and corresponding algorithms are proposed and verified.
π― What it does: Propose the SAFER framework to address the open-set (unknown class) problem in graph-class incremental learning (GCIL), using subspace-aware feature reshaping and task routing under the condition of no replay.
π― What it does: Proposed and implemented SWE-MiniSandbox, a lightweight sandbox without containers, based on Linux namespace and chroot, used for training software engineering agents in reinforcement learning.
Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression
Ruoling Qi (Institute of Artificial Intelligence, China Telecom), Qizhen Weng (Dolby Laboratories)
CodeCompressionOptimizationComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelAuto EncoderText
π― What it does: Propose a training-free low-rank LLM compression framework called Swift-SVD, which utilizes activation-aware closed-form solutions and incremental covariance aggregation to achieve single feature decomposition, supporting dynamic hierarchical compression rate allocation.
π― What it does: This paper derives new generalization upper bounds under the PAC-Bayes framework, extending the theoretical analysis for non-compact symmetric (such as translation, scaling, Lorentz transformation) and non-translation-invariant data distributions, and provides strict generalization advantages of symmetric models in such scenarios;
π― What it does: This paper successfully parses the model's inference process into identifiable hierarchical recursive dynamics by enforcing feature-label permutation symmetry at each layer of the Transformer, and reveals the implementation of a 'coupled feature-label' average drift algorithm;
SynerMedGen: Synergizing Medical Multimodal Understanding with Generation via Task Alignment
Weiren zhao, Cheng Chen (University of Hong Kong)
CodeGenerationData SynthesisRepresentation LearningTransformerMixture of ExpertsVision Language ModelDiffusion modelScore-based ModelAuto EncoderImageMultimodalityBiomedical DataMagnetic Resonance ImagingComputed TomographyPositron Emission Tomography
π― What it does: Proposed the SynerMedGen unified framework, which combines the generation and alignment understanding tasks to achieve collaborative training for medical multimodal understanding and generation.
SynLaD: Latent Diffusion for Generating Synthesizable Molecules Conditioned on 3D Pharmacophore Profiles
Miruna Cretu (University of Cambridge), Colin A Grambow
CodeDrug DiscoveryTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningGraphBiomedical Data
π― What it does: SynLaD learns a unified latent space through a dual-head variational autoencoder (3D geometry decoder + synthetic pathway decoder), and generates novel molecules that meet specified 3D pharmacophore features and are synthetically feasible by utilizing a latent diffusion model in this space.
Synthesizable Molecular Generation via Soft-constrained GFlowNets with Rich Chemical Priors
Hyeonah Kim (Mila - Quebec AI Institute), Alex HernΓ‘ndez-GarcΓa
CodeDrug DiscoveryGraph Neural NetworkTransformerLarge Language ModelReinforcement LearningFlow-based ModelContrastive LearningTextGraphTabular
π― What it does: Proposed the S3-GFN framework, which generates synthetically feasible molecules using a soft-constrained GFlowNet and a pre-trained SMILES language model.
TabICLv2: A Better, Faster, Scalable, and Open Tabular Foundation Model
Jingang QU, Marine Le Morvan (INRIA Saclay)
CodeClassificationData SynthesisOptimizationComputational EfficiencyTransformerMixture of ExpertsDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningTabularBenchmark
π― What it does: Developed TabICLv2, a new table-based model that integrates an improved pre-training process, scalable attention mechanisms, a synthetic data generator, and various structural innovations;
TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention
Si-Yang Liu (Nanjing University), Han-Jia Ye (Nanjing University)
CodeComputational EfficiencyRepresentation LearningTransformerPrompt EngineeringMixture of ExpertsContrastive LearningTabularBenchmark
π― What it does: Proposes TABSWIFT, a lightweight row-wise attention-only Tabular Foundation Model that supports classification and regression, and improves pretraining stability by incorporating learnable register tokens and gated attention, while achieving per-sample adaptive early exiting.
π― What it does: This paper addresses the 'false forgetting' problem in machine learning by proposing an evaluation metric CR based on conformal prediction and an improved CPU framework, which enhances the real forgetting quality of the model.
π― What it does: Propose a target-agnostic training framework FGR, achieving adaptive calibration under distribution shift through frequency-domain low-pass filtering and gradient correction.
Target-Oriented Pretraining Data Selection via Neuron-Activated Graph
Zijun Wang (UC Santa Cruz), Fengze Liu (ByteDance)
CodeExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelText
π― What it does: Studies a method for untrained, interpretable screening and ranking of goal-oriented pre-training data through Neuron-Activated Graph (NAG).
π― What it does: Propose a task-aware exploration method called TEB based on predictive bisimulation metric, which can simultaneously learn task-related representations and generate global exploration rewards in visual reinforcement learning with sparse rewards;
Task-Awareness Improves LLM Generations and Uncertainty
Tim Tomov (Technical University of Munich), Stephan GΓΌnnemann
CodeClassificationGenerationTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsTextRetrieval-Augmented Generation
π― What it does: This paper proposes a method that maps the generation results of large language models (LLMs) into a task-specific latent structure space, and directly performs minimum Bayes risk (MBR) decoding on this space, generating task results that are superior to those produced by traditional decoding methods, while quantifying uncertainty through Bayesian risk.
π― What it does: This paper proposes a continuous learning framework called LoDA based on LoRA, which utilizes task-driven projection energy decomposition to divide the update space into a general subspace and a specific subspace. It constructs a dual-branch LoRA module to achieve knowledge sharing and isolation, and after each task, it achieves feature-level joint optimization through closed-form re-calibration.