ICML 2025 Papers — Page 31
International Conference on Machine Learning · 3257 papers
Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees
Zehong Wang (University of Notre Dame), Yanfang Ye (University of Notre Dame)
Graph Neural NetworkSupervised Fine-TuningContrastive LearningGraph
🎯 What it does: Proposes Task-Trees as a unified learning instance, constructs and pre-trains a Graph Neural Network (GIT) to achieve cross-task and cross-domain graph-based models, and implements domain specialization through instruction fine-tuning.
Towards Learning to Complete Anything in Lidar
Ayça Takmaz (NVIDIA), Aljosa Osep (NVIDIA)
RecognitionObject DetectionSegmentationGenerationTransformerVision Language ModelVideoPoint Cloud
🎯 What it does: A zero-shot LiDAR scene-oriented instance-level shape completion framework (CAL) is proposed, capable of completing and recognizing objects of any category from a single frame of sparse LiDAR point clouds.
Towards Lifelong Model Editing via Simulating Ideal Editor
Yaming Guo (Hong Kong University of Science and Technology), Ying Sun (Hong Kong University of Science and Technology)
TransformerLarge Language ModelText
🎯 What it does: A general framework SimIE is proposed to transfer standard model editing methods to lifelong editing scenarios.
Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond
Chongyu Fan (Michigan State University), Sijia Liu (Michigan State University)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Proposes integrating Sharpness-Aware Minimization (SAM) and other smoothing optimization techniques into the LLM (Large Language Model) zero-shot learning process to enhance the model's robustness against re-learning attacks.
Towards Memorization Estimation: Fast, Formal and Free
Deepak Ravikumar (Purdue University), Kaushik Roy (Purdue University)
ClassificationRecognitionAnomaly DetectionComputational EfficiencyConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: Proposed and validated the Cumulative Sample Loss (CSL) as a fast, overhead-free memorization proxy, theoretically related to memorization of learning time and stability, and used to identify mislabeled and duplicate samples.
Towards Practical Defect-Focused Automated Code Review
Junyi Lu (Institute of Software Chinese Academy of Sciences), Chun Zuo (Sinosoft Company Limited)
AI Code AssistantLarge Language ModelTextChain-of-Thought
🎯 What it does: A complete, defect-oriented automated code review pipeline is proposed to address four major challenges: context capture, key defect detection, false positive reduction, and human-machine collaboration.
Towards Rationale-Answer Alignment of LVLMs via Self-Rationale Calibration
Yuanchen Wu (Shanghai University), Xiaoqiang Li (Shanghai University)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningVision Language ModelMultimodality
🎯 What it does: Proposes the Self-Rationale Calibration (SRC) framework to enhance the reasoning and answer alignment capabilities of LVLM through self-calibration.
Towards Robust Influence Functions with Flat Validation Minima
Xichen Ye (Fudan University), Yifan Chen (Hong Kong Baptist University)
Anomaly DetectionOptimizationData-Centric LearningTransformerSupervised Fine-TuningImageText
🎯 What it does: This paper studies the robustness of the Influence Function in deep networks, pointing out that the sharpness of the validation risk leads to the failure of traditional methods, and proposes a new influence function estimation method based on Flat Validation Minima.
Towards Robustness and Explainability of Automatic Algorithm Selection
Xingyu Wu (Hong Kong Polytechnic University), KC Tan
OptimizationExplainability and InterpretabilityTabularBenchmark
🎯 What it does: An automatic algorithm selection method based on causal graphs (DAG) called DAG-AS is proposed, which predicts the optimal algorithm and provides interpretability by utilizing the conditional distribution between algorithm and problem features.
Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models
Konstantin Donhauser (ETH Zürich), Jason Hartford (University of Manchester)
Explainability and InterpretabilityRepresentation LearningDrug DiscoveryAuto EncoderImageBiomedical Data
🎯 What it does: This paper utilizes sparse dictionary learning to extract interpretable biological concepts, such as cell types, gene interference, and morphological changes, from the internal representations of a basic model (Masked AutoEncoder) for cell microscopy images.
Towards the Causal Complete Cause of Multi-Modal Representation Learning
Jingyao Wang (Institute of Software Chinese Academy of Sciences), Hui Xiong (Hong Kong University of Science and Technology)
Representation LearningContrastive LearningMultimodality
🎯 What it does: Proposed and implemented Causal Complete Cause Regularization (C3R) to learn multimodal representations through causal completeness constraints.
Towards the Efficient Inference by Incorporating Automated Computational Phenotypes under Covariate Shift
chao ying, Jiwei Zhao (University of Wisconsin-Madison)
Domain AdaptationAnomaly DetectionComputational EfficiencyBiomedical DataElectronic Health Records
🎯 What it does: In the semi-supervised learning framework of covariate shift, this study investigates how to effectively integrate Automated Computation of Phenotypes (ACP) into the estimation process to enhance parameter estimation efficiency.
Towards Theoretical Understanding of Sequential Decision Making with Preference Feedback
Simone Drago (Politecnico di Milano), Alberto Maria Metelli (Politecnico di Milano)
OptimizationReinforcement Learning from Human Feedback
🎯 What it does: A theoretical framework for Preference-based Markov Decision Processes (PbMDP) is proposed, and based on this, the compatibility of preferences and multi-dimensional utility functions, the dominance relationship of strategies, and optimality are defined; the construction of compatible utilities and the computational complexity of dominance determination are studied, and a quadratic programming method is provided to approximate non-Markov utilities as Markov rewards along with its error upper bound.
Towards Trustworthy Federated Learning with Untrusted Participants
Youssef Allouah (École Polytechnique Fédérale de Lausanne), John Stephan (École Polytechnique Fédérale de Lausanne)
Federated LearningSafty and PrivacyConvolutional Neural NetworkImage
🎯 What it does: A robust and privacy-preserving algorithm CAFCOR is proposed for decentralized federated learning.
Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Boqi Li (Wuhan University), Weiwei Liu (Wuhan University)
Convolutional Neural NetworkImage
🎯 What it does: Theoretical analysis of a two-layer convolutional neural network is conducted, studying the mechanism of catastrophic forgetting (CF) in incremental continual learning tasks, and providing interpretable conditions; simultaneously, the theory is validated and the effect of replay-based methods on suppressing CF is explored.
Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Xu Wang (Chinese University of Hong Kong), Difan Zou (Hong Kong University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper systematically studies the fine-tuning mechanism of large language models on mathematical tasks through circuit analysis techniques and proposes an improved method based on circuit changes called CircuitLoRA.
Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysis
Christophe Vauthier (University Paris-Saclay), Quentin Mérigot (University Paris-Saclay)
OptimizationPoint Cloud
🎯 What it does: The paper systematically analyzes the gradient dynamics of the sliced Wasserstein distance (SW) as an objective function, particularly focusing on the definition, existence, stability of its critical points, and their impact on the convergence behavior of particle distributions.
Towards Understanding Parametric Generalized Category Discovery on Graphs
Bowen Deng (Sun Yat-Sen University), Chuan Chen (Sun Yat-Sen University)
ClassificationRepresentation LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper proposes the task of Graph General Category Discovery (GGCD) and explores how old class knowledge can assist parameterized GCD methods through theoretical analysis.
Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings
Rong-Xi Tan (Nanjing University), Chao Qian (Nanjing University)
OptimizationTransformerLarge Language ModelContrastive LearningTabular
🎯 What it does: A general offline black-box optimization framework called UniSO is proposed, which utilizes a stringified search space, metadata guidance, and multi-task learning to achieve cross-domain generalization.
Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation
Fanqing Meng (Shanghai Jiao Tong University), Ping Luo (The University of Hong Kong)
GenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextBenchmarkPhysics Related
🎯 What it does: A physical commonsense evaluation benchmark, PhyGenBench, and a corresponding automatic evaluation framework, PhyGenEval, have been constructed to systematically assess the intuitive physical understanding capabilities of text-to-video models.
TRACE Back from the Future: A Probabilistic Reasoning Approach to Controllable Language Generation
Gwen Yidou Weng, Guy Van den Broeck (University of California)
GenerationKnowledge DistillationReinforcement LearningPrompt EngineeringText
🎯 What it does: Proposes the TRACE framework, which implements controllable text generation based on probabilistic reasoning using a one-shot distilled HMM and a lightweight log-linear classifier.
TraceGrad: a Framework Learning Expressive SO(3)-equivariant Non-linear Representations for Electronic-Structure Hamiltonian Prediction
Shi Yin (Artificial Intelligence Institute), Lixin He (University of Science and Technology of China)
Graph Neural NetworkGraphPhysics Related
🎯 What it does: A framework named TraceGrad is proposed, which combines the theory of SO(3) equivariance with a gradient mechanism to predict electronic structure Hamiltonians.
Tracking Most Significant Shifts in Infinite-Armed Bandits
Joe Suk (Columbia University), Jung-hun Kim (CREST ENSAE Paris)
Reinforcement LearningTabular
🎯 What it does: This study investigates an infinite-armed bandit problem where the average reward of actions is initially sampled from a reserve distribution. It presents the first non-parametric optimal regret bounds and relaxes the distribution assumptions.
Tracking The Best Expert Privately
Hilal Asi (Apple), Aadirupa Saha (University of Illinois)
OptimizationSafty and Privacy
🎯 What it does: In the framework of online expert prediction, a differential privacy algorithm is proposed, providing sublinear dynamic return guarantees under dynamic (tracking the optimal expert) conditions against random, oblivious, and adaptive adversaries.
Tractable Transformers for Flexible Conditional Generation
Anji Liu (University of California), Guy Van den Broeck (University of California)
GenerationTransformerLarge Language ModelText
🎯 What it does: Proposes Tractable Transformers (Tracformer), a NAR generation model specifically designed for flexible conditional generation.
Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions
Jaeyeon Kim (Harvard University), Sitan Chen (Harvard University)
GenerationOptimizationTransformerDiffusion modelTextSequential
🎯 What it does: This study explores the trade-off between the vast set of subproblems faced by Masked Diffusion Models (MDM) during training and the flexibility of freely choosing the token order during inference, and proposes a reasoning strategy that enhances performance by adaptively selecting the decoding order.
Training a Generally Curious Agent
Fahim Tajwar (Carnegie Mellon University), Russ Salakhutdinov (Carnegie Mellon University)
Large Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Trained a transferable, generally curious agent (PAPRIKA) by self-generating interactive trajectories across various text decision tasks and using preference optimization to enable LLM to learn general exploration decision strategies.
Training Deep Learning Models with Norm-Constrained LMOs
Thomas Pethick (Ecole Polytechnique Federale de Lausanne), Volkan Cevher (Ecole Polytechnique Federale de Lausanne)
OptimizationConvolutional Neural NetworkTransformerLarge Language ModelImageText
🎯 What it does: A stochastic conditional gradient (SCG) framework based on norm-constrained linear minimization operators (LMO) is proposed for training deep networks, which can be directly applied to unconstrained problems.
Training Diffusion-based Generative Models with Limited Data
Zhaoyu Zhang (Queen's University Belfast), Seán McLoone (Queen's University Belfast)
GenerationData SynthesisDiffusion modelAuto EncoderImage
🎯 What it does: This paper proposes a method for training diffusion models from scratch under conditions of scarce samples, aiming to address the traditional diffusion model's reliance on massive data.
Training Dynamics of In-Context Learning in Linear Attention
Yedi Zhang (Gatsby Computational Neuroscience Unit, University College London), Andrew M Saxe
TransformerTabularOrdinary Differential Equation
🎯 What it does: This paper studies the training dynamics of multi-head linear attention models in context linear regression tasks through gradient flow analysis, comparing two parameterization methods: merged key-query and separated key-query. It reveals their fixed point structures, loss descent processes (single sharp drop or multiple cliff-like descents), and the progressive principal component regression achieved by the model during training.
Training Flexible Models of Genetic Variant Effects from Functional Annotations using Accelerated Linear Algebra
Alan Nawzad Amin, Andrew Gordon Wilson (New York University)
Convolutional Neural NetworkTransformerBiomedical Data
🎯 What it does: Proposes the DeepWAS method, which utilizes accelerated linear algebra to directly maximize the full likelihood, training a deep neural network capable of predicting variant effects from functional annotations;
Training High Performance Spiking Neural Network by Temporal Model Calibration
Jiaqi Yan (Zhejiang University), Gang Pan (Zhejiang University)
ClassificationSpiking Neural NetworkImageText
🎯 What it does: Proposes the Temporal Model Calibration (TMC) method, which utilizes gradient re-scaling in the time dimension (confidence regularization + exponential λ constraint) to enhance the diversity of temporal logit gradients, achieving direct training of SNNs for temporal calibration.
Training Software Engineering Agents and Verifiers with SWE-Gym
Jiayi Pan (University of California Berkeley), Yizhe Zhang (Apple)
AI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: A SWE-Gym training environment was constructed, and it was used to train language models to generate solutions and validators for software engineering tasks, achieving scalability during agent training and inference.
Trajectory Inference with Smooth Schrödinger Bridges
Wanli Hong (New York University), Jonathan Niles-Weed (New York University)
Object TrackingOptimizationComputational EfficiencyPoint CloudTime SeriesSequentialStochastic Differential Equation
🎯 What it does: A smooth Schrödinger bridge method is proposed, replacing traditional Brownian motion with a smooth high-order autoregressive Gaussian process to address noise and path roughness issues in particle trajectory inference.
Trajectory World Models for Heterogeneous Environments
Shaofeng Yin (Tsinghua University), Mingsheng Long (Tsinghua University)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerWorld ModelTime SeriesSequential
🎯 What it does: A unified trajectory world model (TrajWorld) is proposed for different sensor and actuator configurations, and a large-scale trajectory dataset (UniTraj) covering 80 environments is constructed, enabling transfer learning across environments.
Transfer Learning for Nonparametric Contextual Dynamic Pricing
Fan Wang (University of Warwick), Yi Yu (University of Warwick)
Domain AdaptationRecommendation SystemOptimizationTabularFinance Related
🎯 What it does: A non-parametric context dynamic pricing algorithm based on transfer learning, TLDP, is proposed to improve pricing performance in the target domain when source domain data is insufficient.
Transfer Q-Learning with Composite MDP Structures
Jinhang Chai (Princeton University), Lin Yang (UCLA)
OptimizationRobotic IntelligenceReinforcement Learning
🎯 What it does: A new composite MDP framework is proposed, which splits the transition dynamics into a low-rank shared structure and a sparse task-specific variation, and based on this, the UCB-TQL (Upper Confidence Bound Transfer Q-Learning) algorithm is designed to achieve knowledge transfer between multiple tasks.
Transformative or Conservative? Conservation laws for ResNets and Transformers
Sibylle Marcotte (Ecole Normale Supérieure), Gabriel Peyré (CNRS)
Convolutional Neural NetworkTransformerImageText
🎯 What it does: This paper analyzes the structure of Residual Networks (ResNet) and Transformers under the dynamics of gradient flow training, proposes and proves new conservation laws, and verifies that these laws are approximately maintained in actual Stochastic Gradient Descent (SGD) training.
Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimation
He Li (National University of Defense Technology), Wenjing Yang (National University of Defense Technology)
Convolutional Neural NetworkTransformerTime SeriesSequential
🎯 What it does: A Transformer-based spatio-temporal causal inference framework is proposed, capable of estimating counterfactual outcomes under spatio-temporal attributes.
Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries
Huakun Luo (Tsinghua University), Mingsheng Long (Tsinghua University)
MeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Developed the Transolver++ neural PDE solver on large-scale geometric meshes, supporting millions of points.
TransPL: VQ-Code Transition Matrices for Pseudo-Labeling of Time Series Unsupervised Domain Adaptation
Jaeho Kim (Ulsan National Institute of Science and Technology), Seulki Lee (Ulsan National Institute of Science and Technology)
Domain AdaptationTime SeriesElectrocardiogram
🎯 What it does: A pseudo-labeling method named TransPL is designed for unsupervised time series domain adaptation, primarily generating pseudo-labels through the state transition matrix of vector quantization codes.
Tree-Sliced Wasserstein Distance with Nonlinear Projection
Thanh Tran (VinUniversity), Tan Minh Nguyen
GenerationData SynthesisComputational EfficiencyRepresentation LearningAuto EncoderGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: Proposed a tree partition Wasserstein distance based on nonlinear projection (CircularTSW, SpatialTSW, and their spherical variants), and provided a theoretical proof of injectivity and efficient implementation.
Tree-Sliced Wasserstein Distance: A Geometric Perspective
Hoang V. Tran, Tan Minh Nguyen
GenerationOptimizationDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a new measure - Tree-Sliced Wasserstein Distance (TSW-SL), which measures probability distributions by replacing traditional one-dimensional linear projections with tree structures.
TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree
Yu-Yang Qian (Nanjing University), Zhi-Hua Zhou (Nanjing University)
Computational EfficiencyTransformerLarge Language ModelImageText
🎯 What it does: This paper proposes TreeLoRA, a method for efficient continual learning that utilizes a hierarchical gradient similarity tree structure and LoRA adapters.
Triple-Optimistic Learning for Stochastic Contextual Bandits with General Constraints
Hengquan Guo (ShanghaiTech University), Xin Liu (ShanghaiTech University)
Recommendation SystemOptimizationReinforcement LearningTabular
🎯 What it does: Proposes the Optimistic 3 framework for handling general constraints in the contextual multi-armed bandit problem, providing both theoretical and experimental validation.
Trust-Region Twisted Policy Improvement
Joery A. de Vries (Delft University of Technology), Matthijs T. J. Spaan (Delft University of Technology)
Reinforcement Learning
🎯 What it does: This paper proposes the Trust-Region Twisted SMC method, which combines particle filtering-based planning with MCTS ideas to enhance the policy improvement and sample efficiency of online planning in reinforcement learning.
TRUST-VLM: Thorough Red-Teaming for Uncovering Safety Threats in Vision-Language Models
Kangjie Chen (Nanyang Technological University), Tianwei Zhang (Nanyang Technological University)
GenerationAdversarial AttackTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: An automated, iterative multimodal red team framework called TRUST-VLM is proposed for generating and improving text-image inputs to detect security vulnerabilities in visual language models.
Trusted Multi-View Classification with Expert Knowledge Constraints
Xinyan Liang (Shanxi University), Feijiang Li
ClassificationExplainability and InterpretabilityConvolutional Neural NetworkTime SeriesBiomedical Data
🎯 What it does: A trustworthy multi-view classification framework TMCEK based on expert knowledge constraints is proposed, mainly used for sleep stage classification, integrating Gabor convolution kernels to achieve feature-level interpretability, and employing a distribution-aware subjective opinion mechanism to enhance uncertainty estimation.
Trustworthy Machine Learning through Data-Specific Indistinguishability
Hanshen Xiao (Purdue University), G. Edward Suh (NVIDIA)
Federated LearningTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelImageText
🎯 What it does: A Differential Trust framework is proposed, introducing Data-Specific Indistinguishability (DSI) to quantify and ensure the trustworthiness of machine learning models.
TruthFlow: Truthful LLM Generation via Representation Flow Correction
Hanyu Wang (Pennsylvania State University), Jinghui Chen (Pennsylvania State University)
GenerationTransformerLarge Language ModelFlow-based ModelRectified FlowTextOrdinary Differential Equation
🎯 What it does: The TruthFlow method is proposed, which utilizes a flow matching model to learn a specific truth correction vector for each query, and edits the hidden representations of the LLM during inference to enhance the truthfulness of the generated text.
TS-SNN: Temporal Shift Module for Spiking Neural Networks
Kairong Yu (Zhejiang University), Hongwei Wang (Zhejiang University)
ClassificationComputational EfficiencySpiking Neural NetworkImage
🎯 What it does: This paper proposes the Temporal Shift module, which integrates temporal information from the past, present, and future into the same time step through a lightweight time shift operation in SNN, achieving low-energy and high-accuracy classification.
TSP: A Two-Sided Smoothed Primal-Dual Method for Nonconvex Bilevel Optimization
Songtao Lu (Chinese University of Hong Kong)
OptimizationHyperparameter SearchMeta LearningTabular
🎯 What it does: A single-loop stochastic two-sided smoothing primal-dual optimization (TSP) algorithm is proposed to solve non-convex bilevel problems, employing Moreau envelope smoothing and gradient descent/ascent updates simultaneously in both upper and lower levels, capable of converging to KKT points with high probability.
TtBA: Two-third Bridge Approach for Decision-Based Adversarial Attack
Feiyang Wang (Beijing University of Posts and Telecommunications), Gang Chen (Victoria University of Wellington)
Adversarial AttackImage
🎯 What it does: A decision-based black-box adversarial attack method called TtBA is designed and implemented based on bridge direction.
TTFSFormer: A TTFS-based Lossless Conversion of Spiking Transformer
Lusen Zhao (Peking University), Zhaofei Yu (Peking University)
Spiking Neural NetworkTransformerImage
🎯 What it does: Convert the pre-trained Transformer model to TTFS-encoded SNN, achieving a lossless conversion without training.
TuCo: Measuring the Contribution of Fine-Tuning to Individual Responses of LLMs
Felipe Pinto Coelho Nuti (University of Oxford), Joao F. Henriques (University of Oxford)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: An indicator called Tuning Contribution (TuCo) is proposed to quantify the contribution of fine-tuning to a single output during the inference process, and its effectiveness is validated through the decomposition of the residual flow.
TUMTraf VideoQA: Dataset and Benchmark for Unified Spatio-Temporal Video Understanding in Traffic Scenes
Xingcheng Zhou (Technical University of Munich), Alois Knoll
Autonomous DrivingTransformerLarge Language ModelVision Language ModelVideoTextBenchmark
🎯 What it does: Proposed the TUMTraf VideoQA dataset and benchmark for multi-task video understanding in roadside traffic scenes.
Tuning LLM Judge Design Decisions for 1/1000 of the Cost
David Salinas (University of Freiburg), Frank Hutter (University of Freiburg)
OptimizationHyperparameter SearchTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper significantly reduces evaluation costs and improves human consistency by systematically tuning the hyperparameters of the LLM evaluator.
Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimization
Kyurae Kim (University of Pennsylvania), Trevor Campbell (University of British Columbia)
OptimizationTabularStochastic Differential Equation
🎯 What it does: Proposes an online adaptive method based on greedy incremental KL minimization for tuning the path proposal Markov kernel of SML samplers (especially the step size).
Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Lee Cohen (Stanford University), Judy Hanwen Shen (Stanford University)
Large Language ModelText
🎯 What it does: This study investigates the fairness and accuracy of recruitment under the manipulation of large language models (LLMs), proposing the Two-Ticket scheme and its extended n-Ticket scheme to mitigate discrimination caused by differential access to LLMs.
TypyBench: Evaluating LLM Type Inference for Untyped Python Repositories
Honghua Dong (University of Toronto), Xujie Si (University of Toronto)
AI Code AssistantTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes the TYPYBENCH benchmark, designs two new metrics (TYPESIM and TYPECHECK), and evaluates the type inference capabilities of various large language models on 50 high-quality Python repositories.
UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning
Jiawei Zhang (University of Chicago), Bo Li (University of Chicago)
OptimizationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A unified red team framework named UDora is proposed, which can induce LLM agents to perform malicious actions by dynamically hijacking their reasoning processes.
UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models
Xin Xu (Hong Kong University of Science and Technology), Yang Wang (Hong Kong University of Science and Technology)
Large Language ModelTextBenchmarkPhysics Related
🎯 What it does: UGPhysics has been constructed as a benchmark for undergraduate-level physics reasoning, and the MARJ evaluation framework has been proposed.
UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction
Shravan Nayak (Mila - Quebec AI Institute), Sai Rajeswar (ServiceNow Research)
RecognitionObject DetectionRetrievalTransformerVision Language ModelVideoMultimodalityBenchmark
🎯 What it does: Developed UI-Vision, the first offline GUI benchmark covering 83 desktop applications, providing dense annotations of UI elements, layouts, and interaction trajectories;
Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion
Anle Ke (Nanjing University), Zhan Ma (Nanjing University)
RestorationCompressionLarge Language ModelDiffusion modelImage
🎯 What it does: A novel extreme low bitrate image compression framework called ResULIC is proposed, which can achieve high-quality reconstruction under conditions of less than 0.005 bpp.
Ultra-Resolution Adaptation with Ease
Ruonan Yu (National University of Singapore), Xinchao Wang (National University of Singapore)
GenerationData SynthesisSuper ResolutionDiffusion modelImage
🎯 What it does: This paper studies the extension of text-to-image diffusion models to ultra-high resolution (2K/4K) generation under limited data and computational resources, and proposes the URAE method (synthetic data, parameter efficiency, CFG disabled).
UltraTWD: Optimizing Ultrametric Trees for Tree-Wasserstein Distance
Fangchen Yu (Chinese University of Hong Kong), Qiang Sun (Mohamed bin Zayed University of Artificial Intelligence)
RetrievalOptimizationText
🎯 What it does: This paper proposes UltraTWD, an unsupervised framework that can simultaneously optimize tree structure and edge weights, making the tree-Wasserstein distance closer to the 1-Wasserstein distance.
Unbiased Evaluation of Large Language Models from a Causal Perspective
Meilin Chen (Hikvision Research Institute), Jiang Zhu (Hikvision Research Institute)
TransformerLarge Language ModelAgentic AIText
🎯 What it does: An unbiased evaluation framework called Unbiased Evaluator based on a causal perspective is proposed, which conducts theoretical and empirical analysis of the existing data bias and model bias in Agents-as-an-Evaluator, and dynamically evaluates LLMs through BOAT (Bag of Atomic Interventions).
Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Hao Wang (Zhejiang University), Zhouchen Lin (Peking University)
Recommendation SystemTabular
🎯 What it does: This paper proposes the WeaklyRec framework, which utilizes weakly supervised learning to address the issue of missing negative samples in implicit feedback recommendation, and introduces a risk estimation without negative samples.
UncertainSAM: Fast and Efficient Uncertainty Quantification of the Segment Anything Model
Timo Kaiser (Leibniz University Hannover), Bodo Rosenhahn (Leibniz University Hannover)
SegmentationOptimizationComputational EfficiencyHyperparameter SearchImage
🎯 What it does: This paper proposes the UncertainSAM method for rapid and efficient quantification of prediction uncertainty in the Segment Anything Model (SAM), which includes Bayesian approximation and a lightweight USAM MLP estimator.
Uncertainty Estimation for Heterophilic Graphs Through the Lens of Information Theory
Dominik Fuchsgruber (Technical University of Munich), Stephan Günnemann
Anomaly DetectionGraph Neural NetworkGraph
🎯 What it does: A method for estimating node uncertainty based on information theory, called Joint Latent Density Estimation (JLDE), is proposed and validated for heterophilic graphs.
Uncertainty Quantification for LLM-Based Survey Simulations
Chengpiao Huang (Columbia University), Kaizheng Wang (Columbia University)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A questionnaire simulation method based on large language models (LLM) is proposed, which constructs confidence intervals using generated synthetic answers and achieves reliable inference of real population parameters by adaptively selecting the simulated sample size.
Uncertainty-Based Extensible Codebook for Discrete Federated Learning in Heterogeneous Data Silos
Tianyi Zhang (University of Minnesota), Dianbo Liu (National University of Singapore)
Federated LearningImage
🎯 What it does: A federated learning framework based on uncertainty-extended codebooks (UEFL) is proposed, which improves model accuracy and reduces uncertainty by dynamically expanding discrete vector codebooks under multi-data silos.
Unconstrained Robust Online Convex Optimization
Jiujia Zhang (Boston University), Ashok Cutkosky (Boston University)
Optimization
🎯 What it does: This paper studies online learning in the presence of 'damaged' feedback and proposes an algorithm to maintain low regret in unconstrained online convex optimization.
Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning
Rongzhe Wei (Georgia Institute of Technology), Eli Chien
Safty and PrivacyLarge Language ModelText
🎯 What it does: This study investigates the underestimation of privacy risks for minority groups in the context of non-learning evaluations of large language models. It proposes a minority group-aware evaluation framework and validates it through experiments that insert Personally Identifiable Information (PII) Canaries and directly select minority samples.
Understanding and Improving Length Generalization in Recurrent Models
Ricardo Buitrago, Albert Gu (Carnegie Mellon University)
Recurrent Neural NetworkSupervised Fine-TuningText
🎯 What it does: Through systematic experiments and theoretical analysis, this study investigates and explains the mechanism behind the performance decline of recursive models when exceeding the training context length. It proposes the 'Unexplored State Hypothesis' and its validation, followed by several simple training interventions (random noise, fitting noise, state passing, and truncated backpropagation through time) to expand the model's observed state distribution, achieving length generalization from 2k to 128k levels.
Understanding and Mitigating Memorization in Diffusion Models for Tabular Data
Zhengyu Fang (Case Western Reserve University), Jing Li (Case Western Reserve University)
GenerationData SynthesisDiffusion modelTabular
🎯 What it does: This paper explores the phenomenon of memorization in diffusion models for tabular data generation and proposes two data augmentation methods, TabCutMix and TabCutMixPlus, to reduce memorization.
Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes
Dongjae Jeon (Yonsei University), Albert No (Yonsei University)
GenerationData SynthesisOptimizationDiffusion modelScore-based ModelImageOrdinary Differential Equation
🎯 What it does: This study investigates the phenomenon of memorization in the generation process of diffusion models and proposes a geometric framework based on the sharpness of probability distributions to detect and suppress memorization.
Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models
Shuoyuan Wang (Southern University of Science and Technology), Hongxin Wei (Southern University of Science and Technology)
ClassificationDomain AdaptationTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: This study addresses the issue of confidence miscalibration that arises after prompt tuning in CLIP, proposing the Dynamic Outlier Regularization (DOR) method.
Understanding Chain-of-Thought in LLMs through Information Theory
Jean-Francois Ton (ByteDance), Yang Liu (University of California Santa Cruz)
OptimizationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: A framework based on information theory is proposed to evaluate each step of large language models (LLM) in the Chain-of-Thought (CoT) reasoning process without the need for manual annotation of intermediate steps.
Understanding Complexity in VideoQA via Visual Program Generation
Cristobal Eyzaguirre (Stanford University), Pavel Tokmakov (Toyota Research Institute)
RecognitionGenerationTransformerLarge Language ModelVideoBenchmark
🎯 What it does: Utilizing large language models to generate visual programs, and designing the CodePlexity metric based on the structure and content of the programs to conduct data-driven assessments of the difficulty of VideoQA problems, further using this metric to automatically generate a more challenging CodePlex-QA evaluation set.
Understanding Fixed Predictions via Confined Regions
Connor Lawless (Stanford University), Madeleine Udell (Stanford University)
ClassificationOptimizationExplainability and InterpretabilityMixture of ExpertsTabularFinance Related
🎯 What it does: This paper proposes a framework for validating the recourse of linear classifiers by identifying confined regions in the feature space, and implements region-level auditing without sample points.
Understanding Generalization in Quantum Machine Learning with Margins
Tak Hur (Yonsei University), Daniel K. Park (Yonsei University)
ClassificationRecognitionConvolutional Neural NetworkImagePhysics Related
🎯 What it does: This paper proposes a margin-based generalization upper bound to measure the generalization ability of quantum machine learning models, and experimentally verifies the high correlation between margin distribution and generalization error.
Understanding High-Dimensional Bayesian Optimization
Leonard Papenmeier (Lund University), Luigi Nardi (Lund University)
OptimizationTabular
🎯 What it does: This study investigates the gradient vanishing problem in high-dimensional Bayesian optimization and proposes the MSR method based on maximum likelihood estimation initialization, combined with Random Axis-Aligned Subspace Perturbation (RAASP) to enhance performance.
Understanding Input Selectivity in Mamba: Impact on Approximation Power, Memorization, and Associative Recall Capacity
Ningyuan Teresa Huang (Flatiron Institute), Federico Danieli (Apple)
Recurrent Neural NetworkTime SeriesSequential
🎯 What it does: This study investigates the impact of input selectivity in the Mamba architecture on expressiveness, long-term memory, and associative recall capabilities, providing both theoretical proof and experimental validation.
Understanding Mode Connectivity via Parameter Space Symmetry
Bo Zhao (University of California), Rose Yu (University of California)
Optimization
🎯 What it does: By analyzing the connectivity of the minima of neural networks through parameter space symmetry, this paper derives the number of connected components of minima in linear networks, proves that the minima of invertible weight linear networks can be connected through permutation transformations, and presents low-loss curves generated by symmetry while deriving the impact of curvature on linear connectivity.
Understanding Model Ensemble in Transferable Adversarial Attack
Wei Yao (Renmin University of China), Yong Liu (Renmin University of China)
Adversarial AttackConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: This study investigates the theoretical foundations of transferable model ensemble adversarial attacks, introducing concepts such as transferability error, prediction variance, and Rademacher complexity, and provides error decomposition and upper bounds.
Understanding Model Reprogramming for CLIP via Decoupling Visual Prompts
Chengyi Cai (University of Melbourne), Feng Liu
ClassificationRecognitionTransformerPrompt EngineeringContrastive LearningImage
🎯 What it does: This paper proposes an improved method for visual reprogramming on CLIP—Decoupled Visual Prompts (DVP), which decomposes visual prompts into several independent prompts targeting different subsets of descriptors and utilizes a Probability Reweighting Matrix (PRM) to dynamically assess the contribution of each descriptor to category predictions.
Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach
Changdae Oh (University of Wisconsin Madison), Yixuan Li (University of Wisconsin Madison)
Domain AdaptationRepresentation LearningTransformerLarge Language ModelVision Language ModelTextMultimodality
🎯 What it does: This study investigates the robustness of multimodal large language models (MLLM) under distribution shifts, proposing a theoretical framework based on information theory, defining Effective Mutual Information (EMI), and deriving its upper bound on the difference between ID and OOD. Experiments are then conducted across 61 different shift scenarios for validation.
Understanding Nonlinear Implicit Bias via Region Counts in Input Space
Jingwei Li (Tsinghua University), Jingzhao Zhang (Tsinghua University)
ClassificationConvolutional Neural NetworkImage
🎯 What it does: This study investigates the implicit bias of neural networks and proposes to measure the complexity of the model's decision boundary through the counting of connected regions in the input space.
Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods
Yifan HAO, Tong Zhang (University of Illinois Urbana-Champaign)
OptimizationLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Through experimental and theoretical analysis, the phenomenon of overfitting in supervised fine-tuning was studied, demonstrating that model ensembling can simultaneously improve downstream task performance and alleviate forgetting of upstream tasks.
Understanding Sharpness Dynamics in NN Training with a Minimalist Example: The Effects of Dataset Difficulty, Depth, Stochasticity, and More
Geonhui Yoo (Korea Advanced Institute of Science and Technology), Chulhee Yun (Korea Advanced Institute of Science and Technology)
OptimizationImage
🎯 What it does: This study uses a minimal model (deep linear network with one neuron per layer) to investigate the sharpness dynamics during the training of deep neural networks, particularly the phenomenon of progressive sharpening and its relationship with dataset characteristics, network depth, the randomness of the optimizer, and the step size.
Understanding Synthetic Context Extension via Retrieval Heads
Xinyu Zhao (University of Texas at Austin), Greg Durrett (University of Texas at Austin)
Data SynthesisRetrievalTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: This study investigates the effects of using synthetic data for model fine-tuning in long-context retrieval and reasoning tasks, and explains the advantages and disadvantages of synthetic data through retrieval attention heads.
Understanding the difficulties of posterior predictive estimation
Abhinav Agrawal (University of Massachusetts), Justin Domke (University of Massachusetts)
Recommendation SystemTabular
🎯 What it does: This paper conducts an in-depth analysis of the low signal-to-noise ratio (SNR) problem that arises when using simple Monte Carlo to estimate the posterior predictive density (PPD) in approximate inference, and proposes an adaptive importance sampling (LIS) method based on IW-ELBO to significantly improve estimation accuracy.
Understanding the Emergence of Multimodal Representation Alignment
Megan Tjandrasuwita (Massachusetts Institute of Technology), Paul Pu Liang (Massachusetts Institute of Technology)
Representation LearningTransformerVision Language ModelImageTextMultimodality
🎯 What it does: The system compares the occurrence of implicit alignment between explicit alignment methods and independently trained unimodal models in multimodal tasks, and analyzes the relationship between alignment and task performance.
Understanding the Forgetting of (Replay-based) Continual Learning via Feature Learning: Angle Matters
Hongyi Wan (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
Representation LearningConvolutional Neural NetworkImage
🎯 What it does: In the context of continual learning, this paper deeply analyzes the impact of the angle between task signal vectors on the phenomenon of forgetting using feature learning theory, and demonstrates that the replay mechanism alleviates forgetting by expanding the range of 'beneficial forgetting' angles.
Understanding the Kronecker Matrix-Vector Complexity of Linear Algebra
Raphael A Meyer, David Woodruff
🎯 What it does: This paper studies the query complexity of linear algebra problems (such as trace, spectral norm, and zero matrix determination) under a model where matrix-vector multiplication can only be queried through Kronecker structures.
Understanding the Limits of Deep Tabular Methods with Temporal Shift
Haorun Cai, Han-Jia Ye (Nanjing University)
TabularTime SeriesBenchmark
🎯 What it does: This paper investigates the reasons for performance degradation of deep tabular models in the scenario of temporal shift and proposes an improved training protocol and a lightweight temporal embedding method based on Fourier series to enhance the model's ability to capture trend and periodic information.
Understanding the Logic of Direct Preference Alignment through Logic
Kyle Richardson (Allen Institute for AI), Ashish Sabharwal (Allen Institute for AI)
Recommendation SystemOptimizationReinforcement Learning from Human FeedbackText
🎯 What it does: Formalizes the Direct Preference Alignment (DPA) loss function, proposes a logic-based preference structure, and constructs and explores the lattice structure of the loss space by decompiling algorithms to map existing losses into composable symbolic expressions, verifying and discovering new loss variants.
Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism
Aviv Bick (Carnegie Mellon University), Albert Gu (Carnegie Mellon University)
RetrievalKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: This study investigates the gap in 'retrieval capability' between Transformer and State-Space Models (SSM) on multiple-choice MMLU and other knowledge and reasoning benchmarks. It finds that both rely on a small number of Gather-and-Aggregate (G&A) heads to perform retrieval, and the poor performance of SSM's G&A heads leads to a retrieval bottleneck.
Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees
Xin Yu (Pennsylvania State University), Runze Li (Pennsylvania State University)
Federated LearningImageText
🎯 What it does: This paper studies the trade-off between statistical accuracy and communication efficiency in Personalized Federated Learning (PFL), providing the optimal statistical convergence rate and communication complexity under a multi-task/global + local model framework.