ICML 2024 Papers — Page 2
International Conference on Machine Learning · 2610 papers
Accelerating PDE Data Generation via Differential Operator Action in Solution Space
huanshuo dong, Jie Wang (University of Science and Technology of China)
Data SynthesisComputational EfficiencyTabularPhysics RelatedOrdinary Differential Equation
🎯 What it does: The DiffOAS algorithm is proposed, which directly generates PDE training data by generating basis functions in the solution space and applying operators to them, thus eliminating the traditional step of solving large linear systems.
Accelerating Transformer Pre-training with 2:4 Sparsity
Yuezhou Hu (Tsinghua University), Jun Zhu (Tsinghua University)
OptimizationComputational EfficiencyTransformerText
🎯 What it does: By introducing 2:4 sparse training in Transformer pre-training, acceleration was achieved without loss of accuracy.
Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
Haotong Qin (ETH Zurich), Michele Magno (ETH Zurich)
CompressionOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A framework called IR-QLoRA is proposed for LoRA fine-tuning on low-bit quantized LLMs, significantly improving the accuracy of the quantized model.
ACE: Off-Policy Actor-Critic with Causality-Aware Entropy Regularization
Tianying Ji (Tsinghua University), Huazhe Xu
Reinforcement Learning
🎯 What it does: An offline Actor-Critic algorithm ACE is constructed, which uses a causal policy-reward model to evaluate the impact of actions across dimensions on rewards, and weights the policy entropy to achieve more efficient exploration, while introducing a reset mechanism based on gradient dormancy to avoid local optima.
Achieving Lossless Gradient Sparsification via Mapping to Alternative Space in Federated Learning
Do-Yeon Kim (Korea Advanced Institute of Science and Technology), Jaekyun Moon (Korea Advanced Institute of Science and Technology)
OptimizationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an orthogonal transformation that maps gradients to a new space where the gradients are almost all zero, achieving lossless sparsification and significantly reducing the uplink communication volume in federated learning by using Topk or Sparse-Binary compression in this space.
Achieving Margin Maximization Exponentially Fast via Progressive Norm Rescaling
Mingze Wang (Peking University), Lei Wu (Peking University)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: The study investigates the indirect maximization of margin deviation using gradient descent on linearly separable data and proposes a Progressive Re-Normalized Gradient Descent (PRGD) to accelerate margin maximization.
ACM-MILP: Adaptive Constraint Modification via Grouping and Selection for Hardness-Preserving MILP Instance Generation
Ziao Guo (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Data SynthesisOptimizationGraph Neural NetworkAuto EncoderTabular
🎯 What it does: An ACM-MILP framework based on adaptive constraint modification and community detection is proposed to generate mixed-integer linear programming (MILP) instances that maintain the same level of difficulty as the original instances.
ACPO: A Policy Optimization Algorithm for Average MDPs with Constraints
Akhil Agnihotri (University of Southern California), Haipeng Luo (University of Southern California)
OptimizationReinforcement Learning
🎯 What it does: A trust-region policy optimization algorithm ACPO for average reward constrained Markov decision processes (ACMDP) is proposed and implemented, providing theoretical performance and constraint violation guarantees.
Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts
Onur Celik (Karlsruhe Institute of Technology), Gerhard Neumann (Karlsruhe Institute of Technology)
Robotic IntelligenceReinforcement LearningMixture of Experts
🎯 What it does: This paper presents Di-SkilL, a reinforcement learning framework based on mixed experts, designed to automatically learn diverse skills in unknown contextual environments.
Acquisition Conditioned Oracle for Nongreedy Active Feature Acquisition
Michael Valancius (University of North Carolina), Junier Oliva
Recommendation SystemOptimizationReinforcement LearningTabular
🎯 What it does: This paper proposes a non-greedy, deployable active feature acquisition strategy called Acquisition Conditioned Oracle (ACO).
Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations
Jiaqi Zhai (Meta AI), Yu Shi (Meta AI)
Recommendation SystemTransformerSequential
🎯 What it does: This paper proposes a generative recommender (GRs) framework that treats the recommendation task as a sequence transfer task, and designs an efficient Hierarchical Sequence Transfer Unit (HSTU) to achieve a scalable recommendation model.
Activation-Descent Regularization for Input Optimization of ReLU Networks
Hongzhan Yu (University of California), Sicun Gao
OptimizationAdversarial AttackConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: A new input optimization method for ReLU networks is proposed, which explicitly considers the impact of activation pattern changes on the output and guides the input gradient to move in the optimal direction of the activation space through regularization.
Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choice
Masahiro Kato (Mizuho-DL Financial Technology Co., Ltd.), Ryo Inokuchi
Supervised Fine-TuningReinforcement Learning
🎯 What it does: An active adaptive experiment was designed, where both the covariate distribution of the experimental unit and the treatment allocation probability are adjustable, to efficiently estimate the Average Treatment Effect (ATE).
Active Label Correction for Semantic Segmentation with Foundation Models
Hoyoung Kim (POSTECH), Jungseul Ok (POSTECH)
SegmentationAutonomous DrivingImage
🎯 What it does: Utilize a base model to generate pseudo-labels and superpixels, and correct errors in pixel-level semantic segmentation datasets through an active labeling correction framework.
Active Preference Learning for Large Language Models
William Muldrew (University College London), David Barber (University College London)
Recommendation SystemOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A DPO fine-tuning framework based on active preference learning is proposed, which actively selects the most valuable prompts/generation pairs in each iteration and uses LLM as a reviewer to obtain preference labels, followed by model fine-tuning using DPO.
Active Ranking and Matchmaking, with Perfect Matchings
Hafedh El Ferchichi (ENSAE Paris), Vianney Perchet (Criteo AI Lab)
🎯 What it does: Proposed an active sorting and matching algorithm that must complete a perfect matching (each player competes once) in each round;
Active Statistical Inference
Tijana Zrnic (Stanford University), Emmanuel Candes
OptimizationData-Centric LearningTabular
🎯 What it does: A proactive inference method is proposed, which utilizes the prediction uncertainty of machine learning models to actively select data points that need labeling, thereby achieving effective statistical inference under a limited labeling budget.
AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractors
Yucen Wang (Nanjing University), De-Chuan Zhan (Nanjing University)
Robotic IntelligenceReinforcement LearningWorld ModelVideo
🎯 What it does: A separation world model algorithm AD3 based on implicit action inference is designed to handle homogeneous and heterogeneous visual interference in visual control.
Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models
Zalan Fabian (University of Southern California), Mahdi Soltanolkotabi (University of Southern California)
RestorationDiffusion modelAuto EncoderImage
🎯 What it does: A severity encoding and adaptive diffusion sampling framework based on the latent space of autoencoders (Flash-Diffusion) is proposed to automatically adjust the starting time and number of steps of reverse diffusion under different sample difficulties, achieving sample-adaptive reconstruction of inverse problems.
Adapting Pretrained ViTs with Convolution Injector for Visuo-Motor Control
Dongyoon Hwang (KAIST), Jaegul Choo (KAIST)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerReinforcement LearningImage
🎯 What it does: This paper proposes a lightweight module called CoIn, which injects the spatial locality and translational equivariance bias of convolutional layers into the pre-trained Vision Transformer (ViT), enabling ViT to better adapt to visual-motor control tasks.
Adapting Static Fairness to Sequential Decision-Making: Bias Mitigation Strategies towards Equal Long-term Benefit Rate
Yuancheng Xu (University of Maryland), Furong Huang (University of Maryland)
OptimizationReinforcement LearningSequentialFinance Related
🎯 What it does: A long-term fairness metric named ELBERT is proposed, incorporating supply and demand signals into the Markov decision process. Based on this metric, ELBERT-PO (PPO-based Fair Policy Optimization) is designed to reduce long-term bias in continuous decision-making.
Adaptive Accompaniment with ReaLchords
Yusong Wu (Mila Quebec AI Institute), Cheng-Zhi Anna Huang (Mila Quebec AI Institute)
GenerationTransformerReinforcement LearningContrastive LearningAudio
🎯 What it does: An online improvisational accompaniment model named ReaLchords is proposed, which can generate chord accompaniment in real-time for monophonic melodies.
Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning
Tenglong Liu (National University of Defense Technology), Xin Xu (National University of Defense Technology)
Reinforcement LearningAuto EncoderTabularBenchmark
🎯 What it does: This paper proposes an Advantage-guided Adaptive Policy Regularization method (A2PR) to address the issues of excessive conservativeness and out-of-distribution (OOD) overestimation in offline reinforcement learning.
Adaptive Conformal Inference by Betting
Aleksandr Podkopaev (Walmart Global Tech), Kuang-chih Lee (Walmart Global Tech)
Anomaly DetectionOptimizationTime SeriesFinance Related
🎯 What it does: A parameter-free online goodness-of-fit inference method is proposed, which adaptively estimates prediction intervals using a coin betting framework without assuming exchangeability.
Adaptive Feature Selection for No-Reference Image Quality Assessment by Mitigating Semantic Noise Sensitivity
Xudong Li (Xiamen University), Rongrong Ji (Xiamen University)
Knowledge DistillationTransformerImage
🎯 What it does: A no-reference image quality assessment framework QFM-IQM is proposed, which reduces sensitivity to semantic noise through semantic noise matching and quality consistency constraints, and utilizes knowledge distillation to expand the dataset.
Adaptive Group Personalization for Federated Mutual Transfer Learning
Haoqing Xu (Southeast University), Beilun Wang (Southeast University)
Federated LearningTime Series
🎯 What it does: This paper proposes an adaptive grouped personalization method, AdaGrP, within the framework of federated learning for mutual transfer learning, achieving accurate recovery of learnable structures and parameter estimation in environments with concept drift.
Adaptive Hierarchical Certification for Segmentation using Randomized Smoothing
Alaa Anani (Max Planck Institute for Informatics), Mario Fritz (Max Planck Institute for Informatics)
SegmentationAutonomous DrivingImage
🎯 What it does: This paper proposes an adaptive hierarchical randomized smoothing certification method, aimed at pixel-level classification of semantic segmentation models, to provide robustness proofs at multiple semantic levels, thereby reducing rejection rates and enhancing information gain.
Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulation
Ignat Georgiev (Georgia Institute of Technology), Animesh Garg (University of Toronto)
Robotic IntelligenceReinforcement Learning
🎯 What it does: Proposes Adaptive Horizon Actor-Critic (AHAC), a reinforcement learning algorithm that avoids gradient errors caused by contact with rigid dynamics in differentiable simulation environments through adaptive rolling horizon.
Adaptive Observation Cost Control for Variational Quantum Eigensolvers
Christopher J. Anders (Berlin Institute for the Foundations of Learning and Data), Shinichi Nakajima (RIKEN Center for Advanced Intelligence Project)
OptimizationTabularPhysics Related
🎯 What it does: An adaptive observation cost control method (SubsCoRe) is proposed, which reduces the total measurement overhead by dynamically allocating the number of quantum measurements during the subspace minimum optimization (SMO) process of VQE.
Adaptive Online Experimental Design for Causal Discovery
Muhammad Qasim Elahi (Purdue University), Mahsa Ghasemi (Purdue University)
Graph Neural NetworkGraph
🎯 What it does: Proposed a tracking and stopping algorithm for adaptive online experimental design, utilizing limited intervention samples to learn the complete causal graph.
Adaptive Proximal Gradient Methods Are Universal Without Approximation
Konstantinos Oikonomidis (KU Leuven), Panagiotis Patrinos (KU Leuven)
OptimizationTabular
🎯 What it does: This paper proposes a line search-free adaptive proximal gradient method suitable for locally Hölder continuous gradients and proves its convergence without the need for gradient approximation or line search.
Adaptive Robust Learning using Latent Bernoulli Variables
Aleksandr Karakulev (Uppsala University), Prashant Singh (Science for Life Laboratory)
ClassificationOptimizationReinforcement LearningTabular
🎯 What it does: A robust learning algorithm RLVI based on latent Bernoulli variables is proposed, which automatically infers the data contamination ratio and performs parameter estimation within a maximum likelihood framework.
Adaptive Sampling of k-Space in Magnetic Resonance for Rapid Pathology Prediction
Chen-Yu Yen (New York University), Lerrel Pinto (New York University)
ClassificationConvolutional Neural NetworkReinforcement LearningBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This study proposes an adaptive k-space sampling method based on reinforcement learning, ASMR, for disease detection directly from sparse k-space data.
Adaptive Stabilization Based on Machine Learning for Column Generation
Yunzhuang Shen (Australian Artificial Intelligence Institute), Guangquan Zhang (Australian Artificial Intelligence Institute)
OptimizationGraph Neural NetworkSupervised Fine-TuningGraph
🎯 What it does: Utilizing machine learning to predict the optimal dual solution and introducing it as an adaptive penalty term in column generation, forming Adaptive Stabilized Column Generation (ASCG-ML) to accelerate the convergence of dual iterations.
Adaptive Text Watermark for Large Language Models
Yepeng Liu (University of Florida), Yuheng Bu (University of Florida)
GenerationTransformerLarge Language ModelText
🎯 What it does: An adaptive watermarking mechanism is provided for text generated by large language models, ensuring robustness and security without compromising text quality.
Adaptive-Gradient Policy Optimization: Enhancing Policy Learning in Non-Smooth Differentiable Simulations
Feng Gao (Tsinghua University), Yi Wu (Tsinghua University)
OptimizationReinforcement LearningSequential
🎯 What it does: An adaptive analytical gradient method and the AGPO algorithm are proposed to enhance policy learning efficiency by mixing simulation gradients (SG) and Q gradients (QG) in non-smooth differentiable simulations.
Adaptively Learning to Select-Rank in Online Platforms
Jingyuan Wang (New York University), Zhengyuan Zhou (New York University)
Recommendation SystemOptimizationReinforcement LearningTabular
🎯 What it does: This paper proposes a context-based Las Vegas Bandit algorithm UCR, which can dynamically learn and rank K items from N candidate items to maximize user satisfaction.
Adaptively Perturbed Mirror Descent for Learning in Games
Kenshi Abe (CyberAgent), Atsushi Iwasaki (University of ElectroCommunications)
Optimization
🎯 What it does: An Adaptive Perturbation Mirror Descent (APMD) algorithm is proposed, which can achieve convergence to Nash equilibrium in monotonic games with noise at the end of iterations.
AdsorbDiff: Adsorbate Placement via Conditional Denoising Diffusion
Adeesh Kolluru (Carnegie Mellon University), John R. Kitchin (Carnegie Mellon University)
Graph Neural NetworkDiffusion model
🎯 What it does: We propose AdsorbDiff, a conditional denoising diffusion model designed to predict the optimal sites and orientations of adsorbates on catalyst surfaces.
Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment
Chen Zhang (University of Science and Technology of China), Yang Wang (University of Science and Technology of China)
Reinforcement LearningVideo
🎯 What it does: Developed and deployed a DRL agent system called Sh¯ukai for large-scale fighting games, capable of controlling hundreds of characters with a single model and providing trainable opponents for players.
Advancing Dynamic Sparse Training by Exploring Optimization Opportunities
Jie Ji (Clemson University), Xiaolong Ma (Clemson University)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper studies the Dynamic Sparse Training (DST) framework and proposes BiDST, which simultaneously updates weights and masks through a dual-layer optimization.
Adversarial Attacks on Combinatorial Multi-Armed Bandits
Rishab Balasubramanian (Oregon State University), Haoyu Zhao (Princeton University)
Recommendation SystemAdversarial AttackGraphTabular
🎯 What it does: This paper studies the reward poisoning attack in Combinatorial Multi-Armed Bandits (CMAB), proposes the concept of polynomial attackability and provides necessary and sufficient conditions, designs corresponding attack algorithms, and conducts experimental validation in four practical application scenarios.
Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies
Brian R. Bartoldson (Lawrence Livermore National Laboratory), Bhavya Kailkhura (Lawrence Livermore National Laboratory)
Computational EfficiencyAdversarial AttackConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: This study investigates the robustness limits of adversarial attacks under ℓ∞ constraints on CIFAR-10 and proposes a scaling law for adversarial training that includes the quality of synthetic data (FID).
Adversarially Robust Deep Multi-View Clustering: A Novel Attack and Defense Framework
Haonan Huang (Guangdong University of Technology), Qibin Zhao (Guangdong University of Technology)
Adversarial AttackGenerative Adversarial NetworkContrastive LearningMultimodality
🎯 What it does: This study investigates the adversarial attacks and robust defense of deep multi-view clustering models, proposing a GAN-based attack framework and the AR-DMVC-AM defense method based on adversarial training.
Adversarially Robust Hypothesis Transfer Learning
Yunjuan Wang (Johns Hopkins University), Raman Arora (Johns Hopkins University)
OptimizationAdversarial AttackGenerative Adversarial Network
🎯 What it does: This paper studies the robustness issue of hypothesis transfer learning using auxiliary hypotheses in the presence of adversarial attacks and provides theoretical guarantees.
AegisFL: Efficient and Flexible Privacy-Preserving Byzantine-Robust Cross-silo Federated Learning
Dong Chen (Shandong University), Guangwu Xu (Shandong University)
Federated LearningSafty and PrivacyComputational EfficiencyReinforcement LearningImage
🎯 What it does: Designed AegisFL, an efficient and flexible privacy-preserving and Byzantine-robust cross-stacked federated learning framework.
Agent Instructs Large Language Models to be General Zero-Shot Reasoners
Nicholas Crispino (Washington University in St. Louis), Chenguang Wang (Washington University in St. Louis)
ClassificationGenerationKnowledge DistillationTransformerLarge Language ModelAgentic AIPrompt EngineeringTextChain-of-Thought
🎯 What it does: A self-agent has been constructed that first generates task-specific instructions, and then uses these instructions to guide different large language models (such as Vicuna, Llama-2-70b-chat, GPT-3.5 Turbo) for zero-shot reasoning, thereby enhancing multi-task (generation, classification, reasoning) performance.
Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast
Xiangming Gu (Sea AI Lab), Min Lin (Sea AI Lab)
Adversarial AttackTransformerLarge Language ModelVision Language ModelImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: In a multi-agent environment composed of multimodal large language model agents (MLLM), an attack method called 'infectious jailbreak' was studied, demonstrating that a single adversarial image can infect nearly a hundred million agents and induce harmful behavior within only O(log N) rounds of dialogue.
Agent-Specific Effects: A Causal Effect Propagation Analysis in Multi-Agent MDPs
Stelios Triantafyllou (Max Planck Institute for Software Systems), Goran Radanovic (Max Planck Institute for Software Systems)
Graph Neural NetworkReinforcement LearningAgentic AIGraphBiomedical Data
🎯 What it does: This paper introduces the concept of Agent-Specific Effect (ASE) to quantify the causal effect of a certain agent's behavior on outcomes, which is propagated only through other agents in a Multi-Agent Markov Decision Process (MMDP).
Agnostic Interactive Imitation Learning: New Theory and Practical Algorithms
Yichen Li (University of Arizona), Chicheng Zhang (University of Arizona)
Robotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningSequential
🎯 What it does: A new unbiased estimation theory and algorithm is proposed in interactive imitation learning, which can achieve optimal approximation even when the expert policy is not within the learner's policy class.
Agnostic Learning of Mixed Linear Regressions with EM and AM Algorithms
Avishek Ghosh (Indian Institute of Technology Bombay), Arya Mazumdar (University of California San Diego)
Optimization
🎯 What it does: This paper studies the mixed linear regression problem under agnostic conditions, proving that the gradient EM and gradient AM algorithms can converge to the parameters that minimize the corresponding population loss, providing theoretical convergence and error upper bounds.
Agnostic Sample Compression Schemes for Regression
Idan Attias (Ben Gurion University), Menachem Sadigurschi (Ben Gurion University)
CompressionOptimization
🎯 What it does: This paper studies sample compression schemes under agnostic regression, proposing a general α-approximate compression method and providing a linear size exact compression scheme for linear regression under ℓ1 and ℓ∞ loss; it also proves that for other p (1 < p < ∞), constant size exact compression cannot be achieved.
AI Alignment with Changing and Influenceable Reward Functions
Micah Carroll (University of California Berkeley), Anca Dragan (University of California Berkeley)
Recommendation SystemReinforcement Learning
🎯 What it does: This paper studies the situation where user preferences change over time and are easily influenced by AI in the context of AI alignment. It proposes a Dynamic Reward Markov Decision Process (DR-MDP) framework to characterize the relationship between preference changes and influences, and conducts a theoretical analysis of current alignment methods.
AI Control: Improving Safety Despite Intentional Subversion
Ryan Greenblatt (Redwood Research), Fabien Roger
Safty and PrivacyAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: In this paper, the authors propose a 'control evaluation' method and assess various security protocols in the 'APPS backdoor' testing environment to prevent the injection of code backdoors while allowing the use of powerful but untrusted language models.
Ai-sampler: Adversarial Learning of Markov kernels with involutive maps
Evgenii Egorov (Amsterdam Machine Learning Lab University of Amsterdam), Stratis Gavves
OptimizationAdversarial AttackGenerative Adversarial NetworkMultimodality
🎯 What it does: This paper proposes a reversible Markov chain transition kernel (Ai-sampler) constructed based on reversible neural networks. It achieves efficient sampling by adversarially training the kernel to minimize the total variation distance between the target distribution and the chain's stationary distribution.
ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data
Carmen Martin Turrero, Vincent Parret (Sony Semiconductor Solutions Europe)
ClassificationRecognitionTransformerTime Series
🎯 What it does: Proposes ALERT-Transformer, which combines a hybrid pipeline of asynchronous perception and synchronous processing, using a PointNet-based ALERT module to embed event streams in real-time and provide on-demand readable features;
Algorithm and Hardness for Dynamic Attention Maintenance in Large Language Models
Jan van den Brand (Georgia Tech), Tianyi Zhou (University of Southern California)
TransformerLarge Language Model
🎯 What it does: Proposed and analyzed the dynamic maintenance problem of the attention matrix in large language models, providing a data structure that can quickly query attention results after a single entry update;
Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models
Bilgehan Sel (Virginia Tech), Ming Jin (Virginia Tech)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes a new prompting strategy—Algorithm of Thoughts (AoT), which guides large language models (LLMs) to perform deep thinking and exploration within a single query by providing complete algorithm search examples in context;
Algorithmic Stability Unleashed: Generalization Bounds with Unbounded Losses
Shaojie Li (Renmin University of China), Yong Liu (Renmin University of China)
🎯 What it does: This paper proposes a stricter generalization upper bound obtained through algorithm stability in the case of unconstrained loss functions (sub-Weibull distribution);
Align Your Steps: Optimizing Sampling Schedules in Diffusion Models
Amirmojtaba Sabour (NVIDIA), Karsten Kreis (NVIDIA)
GenerationOptimizationDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes an optimization framework for sampling scheduling in diffusion models (Align Your Steps), achieving adaptive scheduling by minimizing the upper bound of sampling error;
Aligned Objective for Soft-Pseudo-Label Generation in Supervised Learning
Ning Xu (Southeast University), Xin Geng (Southeast University)
Meta LearningImage
🎯 What it does: The SEAL framework is proposed, which dynamically optimizes the soft pseudo-label generator through a learnable meta-network and alternately trains with the prediction model to achieve adaptive generation and utilization of soft pseudo-labels in supervised learning.
Aligning Transformers with Weisfeiler-Leman
Luis Müller (RWTH Aachen University), Christopher Morris (RWTH Aachen University)
Drug DiscoveryTransformerGraphBenchmark
🎯 What it does: A scalable theoretical framework for pure Transformers and Weisfeiler-Leman hierarchical alignment is proposed, achieving a strict expressiveness of Transformers that is higher than k-WL for any k while maintaining feasibility.
All-in-one simulation-based inference
Manuel Gloeckler (University of Tübingen), Jakob H. Macke (Max Planck Institute for Intelligent Systems)
TransformerDiffusion modelScore-based ModelTime SeriesStochastic Differential Equation
🎯 What it does: A full-process simulation inference method called Simformer is proposed, which utilizes transformers and probabilistic diffusion models to achieve joint distribution learning of parameters and data, and can quickly generate any conditional distribution (posterior, likelihood, etc.).
Allocation Requires Prediction Only if Inequality Is Low
Ali Shirali (University of California), Moritz Hardt (Max Planck Institute for Intelligent Systems)
Optimization
🎯 What it does: This paper establishes a theoretical framework to compare the benefits of Individual-Level Allocation (ILA) and Unit-Level Allocation (ULA) in contexts of resource scarcity influenced by social structures.
AlphaFold Meets Flow Matching for Generating Protein Ensembles
Bowen Jing (Massachusetts Institute of Technology), Tommi Jaakkola (Massachusetts Institute of Technology)
Protein Structure PredictionFlow-based ModelBiomedical Data
🎯 What it does: Transforming AlphaFold/ESMFold into a flow-matching based generative model AlphaFLOW/ESMFLOW for sampling structural ensembles given a protein sequence.
AlphaZero-Like Tree-Search can Guide Large Language Model Decoding and Training
Ziyu Wan (Shanghai Jiao Tong University), Jun Wang (University College London)
GenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: A tree search framework TS-LLM based on the AlphaZero idea is proposed to guide the generation process of large language models during both inference and training phases.
Ambiguity-Aware Abductive Learning
Hao-Yuan He (Nanjing University), Ming Li (Nanjing University)
ClassificationRecognitionConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A novel inductive reasoning learning framework is proposed—Ambiguity-Aware Abductive Learning (A3BL), which addresses the issue of traditional ABL easily falling into erroneous pseudo-label problems when dealing with ambiguous reasoning results by probabilistically weighting all possible reasoning candidate sets and optimizing using the EM algorithm.
Ameliorate Spurious Correlations in Dataset Condensation
Justin Cui (University of California), Cho-Jui Hsieh (University of California)
Knowledge DistillationData-Centric LearningContrastive LearningImage
🎯 What it does: This study investigates how bias is amplified or suppressed during the process of dataset condensation and proposes a sample reweighting method based on Kernel Density Estimation (KDE) to mitigate bias.
Amend to Alignment: Decoupled Prompt Tuning for Mitigating Spurious Correlation in Vision-Language Models
Jie ZHANG, Zicong Hong (Hong Kong Polytechnic University)
Domain AdaptationTransformerPrompt EngineeringVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: This paper studies a Decoupled Prompt Tuning (CoOPood) framework aimed at alleviating the spurious correlation problem in visual-language models by splitting image features into invariant and spurious, and employing a dual contrastive learning phase to improve out-of-distribution (OOD) generalization performance.
Amortized Equation Discovery in Hybrid Dynamical Systems
Yongtuo Liu (University of Amsterdam), Stratis Gavves
Graph Neural NetworkTime SeriesOrdinary Differential Equation
🎯 What it does: This paper studies the equation discovery problem in hybrid dynamical systems and proposes an end-to-end AMORE framework that integrates mode classification, equation learning, and switching behavior, extending it to multi-object scenarios (AMORE-MIO).
Amortized Variational Deep Kernel Learning
Alan L. S. Matias (Federal University of Ceara), Diego Mesquita
ClassificationOptimizationGraph Neural NetworkImageGraphTabular
🎯 What it does: This paper proposes a new model AVDKL that combines Gaussian processes with deep neural networks, utilizing scalable variational inference to generate local inducing point distributions at each input location, thereby alleviating the issues of over-correlation and overfitting in traditional DKL.
Amortizing Pragmatic Program Synthesis with Rankings
Yewen Pu (Autodesk AI Research), Daniel Fried (Carnegie Mellon University)
OptimizationComputational EfficiencyKnowledge DistillationTabular
🎯 What it does: Proposes an accelerated method for approximating RSA (Rational Speech Acts) program synthesis through global ranking.
AMPA: Adaptive Mixed Precision Allocation for Low-Bit Integer Training
Li Ding (Shanghai Jiao Tong University), Hongkai Xiong (Shanghai Jiao Tong University)
Convolutional Neural NetworkTransformerImageText
🎯 What it does: An adaptive mixed precision allocation framework (AMPA) suitable for low-bit integer training is proposed.
An amortized approach to non-linear mixed-effects modeling based on neural posterior estimation
Jonas Arruda (University of Bonn), Jan Hasenauer (University of Bonn)
OptimizationDrug DiscoveryRecurrent Neural NetworkFlow-based ModelTime SeriesBiomedical DataStochastic Differential Equation
🎯 What it does: A method based on neural posterior estimation is proposed for parameter inference in nonlinear mixed effects (NLME) models, enabling rapid inference across multiple population models and datasets without the need to rerun simulations.
An Analysis of Linear Time Series Forecasting Models
William Toner (University of Edinburgh), Luke Nicholas Darlow (Huawei Research Centre)
Time Series
🎯 What it does: A rigorous mathematical analysis of various mainstream linear time series forecasting models (DLinear, FITS, RLinear, NLinear, etc.) is conducted, proving that they are functionally equivalent to ordinary linear regression. Experimental validation shows that these models converge to the same or extremely similar solutions under the same data.
An Effective Dynamic Gradient Calibration Method for Continual Learning
Weichen Lin (University of Science and Technology of China), Hu Ding (University of Science and Technology of China)
ClassificationImage
🎯 What it does: This paper proposes a Dynamic Gradient Calibration (DGC) method to alleviate the problem of catastrophic forgetting in continual learning by calibrating gradients.
An Efficient Maximal Ancestral Graph Listing Algorithm
Tian-Zuo Wang (Nanjing University), Zhi-Hua Zhou (Nanjing University)
OptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: This paper proposes a novel, non-violent enumeration algorithm for listing all maximal ancestral graphs (MAGs) within a given Markov equivalence class, given a partial ancestral graph (PAG).
An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems
Hitesh Tulsiani (Amazon), Björn Hoffmeister
RecognitionKnowledge DistillationTransformerContrastive LearningTextAudio
🎯 What it does: A two-stage self-learning framework is proposed, which distills a teacher model with explicit and implicit dialogue context to a single-sentence real-time model to improve speech recognition performance.
An Embodied Generalist Agent in 3D World
Jiangyong Huang (Beijing Institute for General Artificial Intelligence), Siyuan Huang (Beijing Institute for General Artificial Intelligence)
Robotic IntelligenceTransformerLarge Language ModelVision Language ModelVision-Language-Action ModelTextMultimodalityPoint CloudChain-of-Thought
🎯 What it does: A generalized posture agent named LEO is proposed, capable of perceiving, attributing, reasoning, planning, and executing actions in a 3D environment, utilizing LLM combined with 3D vision and language alignment and instruction tuning to achieve multimodal tasks.
An Empirical Examination of Balancing Strategy for Counterfactual Estimation on Time Series
Qiang Huang (Jilin University), Yan Liu (University of Southern California)
Recurrent Neural NetworkTransformerContrastive LearningTime SeriesElectronic Health Records
🎯 What it does: This paper systematically evaluates the effectiveness of various balancing strategies (Adversarial Gradient Reversal, Domain Confusion, Propensity Score Contrastive Learning) used in counterfactual estimation of time series through extensive experiments, exploring the trade-off between balancing and prediction as well as performance under different bias, historical length, and distribution shift scenarios.
An Empirical Study Into What Matters for Calibrating Vision-Language Models
Weijie Tu (Australian National University), Tom Gedeon (Curtin University)
ClassificationDomain AdaptationTransformerVision Language ModelImageMultimodality
🎯 What it does: Conducted systematic experiments on the confidence calibration of 35 Vision-Language models under different datasets, label sets, hierarchies, sample sizes, and distribution differences, evaluating the effectiveness of post-processing methods such as temperature scaling, splines, and histogram binning, and verifying that effective calibration can still be achieved using a small number of samples or synthetic data.
An Empirical Study of Realized GNN Expressiveness
Yanbo Wang (Peking University), Muhan Zhang (Peking University)
Graph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper systematically evaluates the performance of 23 types of GNNs in practical expressiveness by constructing a novel expressive dataset BREC and proposing the RPC evaluation framework.
An Explicit Frame Construction for Normalizing 3D Point Clouds
Justin Baker (University of Utah), Bao Wang (University of Utah)
ClassificationOptimizationPoint Cloud
🎯 What it does: A direct, data-independent framework construction and normalization algorithm is proposed, which can achieve error-free consistent alignment for any 3D point cloud.
An Image is Worth Multiple Words: Discovering Object Level Concepts using Multi-Concept Prompt Learning
Chen Jin (AstraZeneca), Philip Alexander Teare (AstraZeneca)
Object DetectionGenerationTransformerPrompt EngineeringDiffusion modelContrastive LearningImageBiomedical Data
🎯 What it does: A framework called Multi-Concept Prompt Learning (MCPL) is proposed, which utilizes a single image and its corresponding natural language description (containing multiple nouns and adjectives) to learn updatable word vectors on a frozen text-image diffusion model, achieving the discovery and local editing of multiple object concepts.
An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks
Zhifa Ke (Peking University), Junyu Zhang (National University of Singapore)
Reinforcement Learning
🎯 What it does: This paper presents a finite-time convergence analysis of neural network TD learning under non-i.i.d. Markov sampling, providing a sampling complexity of O(ε⁻¹) and extending the method to minimax Q-learning in zero-sum games.
An Independence-promoting Loss for Music Generation with Language Models
Jean-Marie Lemercier (Universität Hamburg), Alexandre Défossez
GenerationData SynthesisTransformerAuto EncoderAudio
🎯 What it does: This study investigates a maximum mean discrepancy (MMD)-based independence regularization loss to enhance the token quality of multi-codebook audio segmenters (auto-encoders) in music generation language models.
An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network
Taeyoung Kim (Korea Advanced Institute of Science and Technology), Hongseok Yang (Korea Advanced Institute of Science and Technology)
TabularOrdinary Differential Equation
🎯 What it does: This paper analyzes the convergence behavior of multilayer perceptrons (MLP) and their input-output Jacobian matrix in the infinite width limit, and derives the linear first-order ODE of the training dynamics along with the corresponding analytical solution within the Jacobian Regularization training framework.
An Information Theoretic Approach to Interaction-Grounded Learning
Xiaoyan Hu (Chinese University of Hong Kong), Ho-fung Leung
ClassificationOptimizationReinforcement LearningImage
🎯 What it does: This paper proposes an information-theoretic method VI-IGL to address the missing reward problem in Interaction-Grounded Learning by learning a reward decoder.
An Information-Theoretic Analysis of In-Context Learning
Hong Jun Jeon (Stanford University), Benjamin Van Roy (Stanford University)
Meta LearningTransformerSequential
🎯 What it does: This paper proposes a set of information-theoretic tools to analyze the errors in meta-learning from sequences and provides a general decomposition formula for the error of the Bayes optimal predictor; subsequently, this framework is applied in the Transformer environment to derive an upper bound on the error of sequences generated by sparse mixture Transformers, thereby providing a theoretical explanation for in-context learning (ICL).
An Interpretable Evaluation of Entropy-based Novelty of Generative Models
Jingwei Zhang (Chinese University of Hong Kong), Farzan Farnia (Chinese University of Hong Kong)
GenerationExplainability and InterpretabilityDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a kernel-based spectral method for assessing the pattern-level novelty of generative models relative to a reference distribution, and introduces the Kernel-Based Entropic Novelty (KEN) score, which quantifies and explains newly emerging patterns.
An Intrinsic Vector Heat Network
Alexander Gao (Roblox Research), Hsueh-Ti Derek Liu (Roblox Research)
GenerationMesh
🎯 What it does: A neural network based on vector heat diffusion is proposed, capable of learning tangent vector fields on the surface of a two-dimensional manifold embedded in three dimensions, and applying it to practical tasks such as quadrilateral mesh generation.
An Iterative Min-Min Optimization Method for Sparse Bayesian Learning
Yasen Wang (Huazhong University of Science and Technology), ye yuan
OptimizationTabularTime Series
🎯 What it does: An iterative Min-Min optimization method based on CCCP is proposed to maximize the marginal likelihood function of Sparse Bayesian Learning (SBL), and a global convergence proof is provided.
An LLM Compiler for Parallel Function Calling
Sehoon Kim (University of California Berkeley), Amir Gholami (University of California Berkeley)
Recommendation SystemComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper presents LLMCompiler, a framework based on compiler principles that automatically decomposes LLM tasks into a function call graph with dependencies and implements parallel execution, significantly reducing inference latency and cost.
An Online Optimization Perspective on First-Order and Zero-Order Decentralized Nonsmooth Nonconvex Stochastic Optimization
Emre Sahinoglu (Northeastern University), Shahin Shahrampour (Northeastern University)
OptimizationTabular
🎯 What it does: The ME-DOL algorithm is proposed, studying the online optimization perspective of distributed non-convex non-smooth stochastic optimization;
An Unsupervised Approach for Periodic Source Detection in Time Series
Berken Utku Demirel (ETH Zurich), Christian Holz (ETH Zurich)
Anomaly DetectionOptimizationConvolutional Neural NetworkRecurrent Neural NetworkContrastive LearningTime SeriesBiomedical DataElectrocardiogram
🎯 What it does: A completely unsupervised periodic detection method is proposed, utilizing regularized training that does not require labels or specific augmentation.
Analysis for Abductive Learning and Neural-Symbolic Reasoning Shortcuts
Xiao-Wen Yang (Nanjing University), Zhi-Hua Zhou (Nanjing University)
Autonomous DrivingImage
🎯 What it does: This paper addresses the reasoning shortcut problem in neural symbolic learning (NeSy) and argument-based learning (ABL), proposing a theoretical framework to quantify shortcut risks and provide mitigation strategies, while experimentally validating their effectiveness.
Analyzing $D^\alpha$ seeding for $k$-means
Etienne Bamas (ETH AI Center), Ola Svensson (EPFL)
OptimizationImage
🎯 What it does: This paper provides a theoretical analysis of Dα sampling (α>2) in the initialization of k-means and proves that it can achieve constant or logarithmic level approximation guarantees across various clustering instances.
AND: Audio Network Dissection for Interpreting Deep Acoustic Models
Tung-Yu Wu (National Taiwan University), Tsui-Wei Weng (University of California San Diego)
RecognitionExplainability and InterpretabilityLarge Language ModelAudio
🎯 What it does: This paper proposes AND (Audio Network Dissection), an automated framework based on LLM for generating natural language descriptions of neurons in deep audio models and for concept identification.
Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary, Physical and Geometric Constraints
Tian Zhu (Institute of Computing Technology, Chinese Academy of Sciences), Haicang Zhang (Institute of Computing Technology, Chinese Academy of Sciences)
Drug DiscoveryProtein Structure PredictionLarge Language ModelDiffusion modelScore-based ModelBiomedical Data
🎯 What it does: Designing the variable region (CDR) sequences and structures of antibodies through a score-based diffusion model combined with evolutionary, physical, and geometric constraints.
Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs
Yeonhong Park (Seoul National University), Jae W. Lee (Seoul National University)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: The Any-Precision LLM scheme utilizes a single n-bit quantized model to generate multiple low-bit-width (3-n bit) sub-models, enabling low-cost deployment of LLMs of various sizes, and develops a dedicated GPU engine to support arbitrary bit-width inference.