ICML 2023 Papers — Page 10
International Conference on Machine Learning · 1828 papers
Learning to Incentivize Information Acquisition: Proper Scoring Rules Meet Principal-Agent Model
Siyu Chen (Yale University), Zhuoran Yang (Yale University)
Reinforcement Learning
🎯 What it does: This paper presents an online learning problem under the principal-agent framework, utilizing appropriate scoring rules to incentivize agents to acquire information, and provides the implementation of the OSRL-UCB algorithm.
Learning to Initiate and Reason in Event-Driven Cascading Processes
Yuval Atzmon (NVIDIA Research), Gal Chechik (Bar Ilan University)
Graph Neural NetworkSequential
🎯 What it does: A new supervised learning framework called Cascade is proposed, allowing agents to achieve a given semantic goal through a single intervention (modifying the initial state) after observing an event cascade in a dynamic system.
Learning to Jump: Thinning and Thickening Latent Counts for Generative Modeling
Tianqi Chen (University of Texas at Austin), Mingyuan Zhou (University of Texas at Austin)
GenerationData SynthesisDiffusion modelAuto EncoderImageText
🎯 What it does: A learning to jump framework is proposed, which constructs a deep generative model capable of generating high-quality non-negative sparse data using the dilution and thickening processes of Poisson counting.
Learning to Learn from APIs: Black-Box Data-Free Meta-Learning
Zixuan Hu (Tsinghua University), Dacheng Tao (University of Sydney)
Knowledge DistillationMeta LearningImage
🎯 What it does: This paper proposes a BiDf-MKD framework that achieves black-box data-free meta-learning (DFML) tasks with only an inference interface, a lack of training data, and an uncertain model structure;
Learning to Maximize Mutual Information for Dynamic Feature Selection
Ian Connick Covert, Su-In Lee (University of Washington)
OptimizationReinforcement LearningImageTabular
🎯 What it does: A dynamic feature selection framework based on Conditional Mutual Information (CMI) is proposed, and a policy network is learned through amortized optimization to directly select the most informative features at each step.
Learning to Optimize Differentiable Games
Xuxi Chen (University of Texas), Zhangyang Wang (University of Texas)
GenerationOptimizationRecurrent Neural NetworkGenerative Adversarial NetworkSequential
🎯 What it does: A framework called L2PG based on Learning to Optimize is proposed to solve stable fixed points of differentiable games, mainly applied to quadratic games and GANs.
Learning to Suggest Breaks: Sustainable Optimization of Long-Term User Engagement
Eden Saig (Technion Israel Institute of Technology), Nir Rosenfeld (Technion Israel Institute of Technology)
Recommendation SystemOptimizationSequentialOrdinary Differential Equation
🎯 What it does: This paper proposes a recommendation system interruption strategy learning framework based on the Lotka-Volterra dynamic model, aiming to maximize long-term user engagement.
Learning Unforeseen Robustness from Out-of-distribution Data Using Equivariant Domain Translator
Sicheng Zhu (University of Maryland), Sanghyun Hong (Oregon State University)
Domain AdaptationConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A two-stage algorithm is proposed, which learns the unforeseen robustness of the target domain using unseen transformed data from the source domain through an equivariant domain translator, and performs consistency regularization on the translated data to enhance the model's robustness to unknown transformations.
Learning Unnormalized Statistical Models via Compositional Optimization
Wei Jiang (Nanjing University), Lijun Zhang (Nanjing University)
GenerationOptimizationContrastive LearningImage
🎯 What it does: Proposes to transform the maximum likelihood objective into a stochastic combinatorial optimization through noise distribution, designing a single-loop stochastic algorithm to directly learn the non-normalized model.
Learning useful representations for shifting tasks and distributions
Jianyu Zhang (New York University), Leon Bottou
Domain AdaptationRepresentation LearningConvolutional Neural NetworkTransformerSupervised Fine-TuningContrastive LearningImage
🎯 What it does: The study utilizes diversified feature representations generated from multiple trainings (with different random seeds) and concatenation (CAT) in multi-distribution and multi-task scenarios, as well as a two-stage fine-tuning process, to enhance the richness and robustness of the representations.
Learning-augmented private algorithms for multiple quantile release
Mikhail Khodak (Carnegie Mellon University), Sergei Vassilvitskii (Google Research)
OptimizationSafty and PrivacyTabular
🎯 What it does: This paper proposes applying the learning-enhanced (algorithms with predictions) framework to differential privacy multi-quantile publishing, utilizing external information (public data or historical data) to construct priors, thereby obtaining error bounds related to prediction quality, and providing a trade-off between robustness and consistency; it also demonstrates improvements over traditional non-prior methods both theoretically and experimentally.
Learning-Rate-Free Learning by D-Adaptation
Aaron Defazio (Meta AI), Konstantin Mishchenko (CNRS)
Recommendation SystemOptimizationTransformerSupervised Fine-TuningImageTextTabular
🎯 What it does: A D-Adaptation method based on dual averaging is proposed, achieving gradient descent and Adam without learning rate hyperparameters and with automatic adjustment, reaching optimal convergence rates on convex Lipschitz functions.
LegendreTron: Uprising Proper Multiclass Loss Learning
Kevin H Lam, Richard Nock (Google Research)
ClassificationOptimizationTabular
🎯 What it does: The LEGENDRETRON algorithm is proposed, which simultaneously learns feasible proper loss functions and probability estimates in multi-class problems.
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Chen Liang (Georgia Institute of Technology), Tuo Zhao
CompressionKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: The paper proposes a two-stage compression method called TED (Task-aware Layer-wise Distillation), which learns task-relevant filters between each layer of the teacher model and the student model, and then achieves knowledge distillation by aligning the filtered representations.
LESS-VFL: Communication-Efficient Feature Selection for Vertical Federated Learning
Timothy Castiglia (Rensselaer Polytechnic Institute), Stacy Patterson (Rensselaer Polytechnic Institute)
Federated LearningComputational EfficiencyTabularBiomedical DataElectronic Health Records
🎯 What it does: A method for achieving communication-efficient feature selection in vertical federated learning is proposed.
LESSON: Learning to Integrate Exploration Strategies for Reinforcement Learning via an Option Framework
Woojun Kim (Korea Advanced Institute of Science and Technology), Youngchul Sung (Korea Advanced Institute of Science and Technology)
Reinforcement Learning
🎯 What it does: A framework named LESSON is proposed, which utilizes an option model to automatically select and integrate multiple exploration strategies to achieve a balance between exploration and exploitation that changes with the learning phase.
LEVER: Learning to Verify Language-to-Code Generation with Execution
Ansong Ni (Yale University), Xi Victoria Lin (Meta AI)
GenerationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningTextTabular
🎯 What it does: The LEVER method is proposed, which utilizes a trained verifier to validate and reorder programs generated by large code language models (code-LLM) based on program execution results, thereby improving the quality of language-to-code generation.
Leveraging Demonstrations to Improve Online Learning: Quality Matters
Botao Hao (Deepmind), Zheng Wen
OptimizationReinforcement LearningTabular
🎯 What it does: This paper studies how to utilize offline demonstration data to enhance the regret of online learning (Thompson Sampling) in multi-armed bandits, proposing an informed TS based on the competence level of experts and providing both theoretical and empirical proofs.
Leveraging Label Non-Uniformity for Node Classification in Graph Neural Networks
Feng Ji (Nanyang Technological University), Wee Peng Tay (Nanyang Technological University)
ClassificationGraph Neural NetworkGraph
🎯 What it does: The study utilizes GNN to predict the non-uniformity of logits to capture graph structural information, and improves node classification through this metric.
Leveraging Offline Data in Online Reinforcement Learning
Andrew Wagenmaker (University of Washington), Aldo Pacchiano (Broad Institute of MIT and Harvard)
Reinforcement LearningTabular
🎯 What it does: Proposed and studied the FineTuneRL framework, which efficiently learns ε-optimal policies by combining offline data with online interaction.
Leveraging Proxy of Training Data for Test-Time Adaptation
Juwon Kang (POSTECH), Suha Kwak (POSTECH)
Domain AdaptationKnowledge DistillationContrastive LearningImage
🎯 What it does: A framework for Test-Time Adaptation (TTA) using training data proxies is proposed, which utilizes a small number of synthetic images and inter-class relationships as two types of lightweight proxies to achieve adaptive training for the target domain.
Lifelong Language Pretraining with Distribution-Specialized Experts
Wuyang Chen (University of Texas at Austin), Claire Cui (Google)
Large Language ModelMixture of ExpertsText
🎯 What it does: This study investigates lifelong pre-trained language models and proposes the Lifelong-MoE method, which utilizes a scalable Mixture-of-Experts model to dynamically add experts and freeze old ones, achieving pre-training under continuous data distributions without forgetting.
Likelihood Adjusted Semidefinite Programs for Clustering Heterogeneous Data
Yubo Zhuang (University of Illinois), Yun Yang (University of Illinois)
OptimizationImage
🎯 What it does: A likelihood adjustment-based semi-definite programming (iLA-SDP) algorithm is proposed to directly maximize the complete likelihood of observed data in high-dimensional data with covariance heterogeneity, thereby achieving accurate recovery of clustering labels.
Linear Causal Disentanglement via Interventions
Chandler Squires (Broad Institute of MIT and Harvard), Caroline Uhler (Broad Institute of MIT and Harvard)
Biomedical Data
🎯 What it does: This paper studies causal separation in linear latent variable causal models and linear observation mixed mappings by performing a perfect intervention on each latent variable, providing identifiability theory and constructive algorithms;
Linear CNNs Discover the Statistical Structure of the Dataset Using Only the Most Dominant Frequencies
Hannah Pinson (Vrije Universiteit Brussel), Vincent Ginis (Harvard John A. Paulson School of Engineering and Applied Sciences)
Convolutional Neural NetworkImageOrdinary Differential Equation
🎯 What it does: Analyzes the learning dynamics of a two-layer linear CNN during the gradient descent process and finds that it can reveal the statistical structure of the dataset through the dominant frequency components.
Linear optimal partial transport embedding
Yikun Bai (Vanderbilt University), Soheil Kolouri (Vanderbilt University)
OptimizationComputational EfficiencyPoint Cloud
🎯 What it does: A linear optimal local transport (LOPT) embedding is proposed, which linearizes the optimal transport (OPT) to reduce computational complexity.
Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trains
Matthew Dowling (Stony Brook University), Il Memming Park (Champalimaud Research)
Time SeriesSequential
🎯 What it does: A linear time latent Gaussian process model, cvHM, is proposed for inferring latent trajectories from neural spike recordings.
Linearly Constrained Bilevel Optimization: A Smoothed Implicit Gradient Approach
Prashant Khanduri (Wayne State University), Mingyi Hong (University of Minnesota)
OptimizationConvolutional Neural NetworkImageStochastic Differential Equation
🎯 What it does: This paper studies and solves the lower-level problem of a linear constrained, strongly convex bilevel optimization problem, proposing an implicit gradient method based on perturbation smoothing.
Linkless Link Prediction via Relational Distillation
Zhichun Guo (University of Notre Dame), Tong Zhao (Snap Inc.)
Computational EfficiencyKnowledge DistillationGraph Neural NetworkGraphBenchmark
🎯 What it does: This paper proposes a method for efficiently predicting links by transferring knowledge from Graph Neural Networks (GNNs) to Multi-Layer Perceptrons (MLPs) through relation distillation.
LinSATNet: The Positive Linear Satisfiability Neural Networks
Runzhong Wang (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
OptimizationReinforcement LearningTabularTime SeriesFinance Related
🎯 What it does: A differentiable LinSAT layer is proposed to satisfy positive linear constraints in a single pass and achieve end-to-end solving of neural networks for constraint problems.
LipsNet: A Smooth and Robust Neural Network with Adaptive Lipschitz Constant for High Accuracy Optimal Control
Xujie Song (Tsinghua University), Xiaoming Simon Wang (Didi Chuxing)
OptimizationReinforcement Learning
🎯 What it does: A neural network structure named LipsNet is proposed, which controls the Lipschitz constant through Multi-dimensional Gradient Normalization (MGN) to achieve action smoothing in RL control strategies.
LIV: Language-Image Representations and Rewards for Robotic Control
Yecheng Jason Ma (University of Pennsylvania), Dinesh Jayaraman (University of Pennsylvania)
Robotic IntelligenceReinforcement LearningContrastive LearningVideoTextMultimodality
🎯 What it does: A Language-Image Value Learning (LIV) framework is proposed, which jointly trains visual-language representations and reward functions, pre-trained on a large-scale human video dataset, and then fine-tuned on a small-scale robot dataset for zero-shot reward prediction and control policy learning.
Live in the Moment: Learning Dynamics Model Adapted to Evolving Policy
Xiyao Wang (University of Maryland), Furong Huang (University of Maryland)
Reinforcement LearningSequential
🎯 What it does: This paper proposes a new dynamic model learning method—Policy-adapted Dynamics Model Learning (PDML), which dynamically adjusts the historical policy mixture distribution when generating samples for policy learning, making the learned dynamic model more aligned with the continuously evolving policy.
Local Optimization Achieves Global Optimality in Multi-Agent Reinforcement Learning
Yulai Zhao (Princeton University), Jason D. Lee (Princeton University)
OptimizationReinforcement Learning
🎯 What it does: A multi-agent PPO algorithm is proposed, and it is proven to converge to a global optimal strategy in fully cooperative Markov games, with a sub-linear convergence rate.
Local Vertex Colouring Graph Neural Networks
Shouheng Li (Australian National University), Qing Wang (Australian National University)
ClassificationGraph Neural NetworkGraph
🎯 What it does: This paper proposes a Local Vertex Colouring (LVC) scheme based on graph search and designs a new Search-Guided Graph Neural Network (SGN) based on this scheme. By coloring the search trees generated by BFS/DFS, it extends the expressiveness of 1-WL and can solve problems such as biconnectivity and shortcut graphs that traditional MPNNs cannot distinguish.
Locally Regularized Neural Differential Equations: Some Black Boxes were meant to remain closed!
Avik Pal (Massachusetts Institute of Technology), Christopher Vincent Rackauckas
TabularBiomedical DataElectrocardiogramStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes regularizing the local error of adaptive solvers when training neural differential equations, allowing the model to adaptively select dynamics that are easier to integrate during the learning process.
Long Horizon Temperature Scaling
Andy Shih (Stanford University), Stefano Ermon (Stanford University)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageText
🎯 What it does: Proposes Long Horizon Temperature Scaling (LHTS) to perform temperature scaling on the joint distribution of generative models in a non-greedy manner, improving sampling quality.
Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labels
Min-Kook Suh (Seoul National University), Seung-Woo Seo (Seoul National University)
ClassificationRecognitionContrastive LearningImage
🎯 What it does: A long-tail recognition framework based on mutual information maximization is proposed, unifying contrastive learning and logit adjustment into the Gaussian Mixture Likelihood loss.
Long-Term Rhythmic Video Soundtracker
Jiashuo Yu (Shanghai Artificial Intelligence Laboratory), Yu Qiao (Shanghai Artificial Intelligence Laboratory)
GenerationData SynthesisRecurrent Neural NetworkDiffusion modelVideoMultimodalityAudio
🎯 What it does: Generating long audio tracks synchronized with video rhythm (video-audio synthesis)
LongCoder: A Long-Range Pre-trained Language Model for Code Completion
Daya Guo (Sun Yat-sen University), Julian McAuley (University of California, San Diego)
AI Code AssistantTransformerLarge Language ModelText
🎯 What it does: A code completion model called LongCoder is proposed for long code contexts, utilizing a sparse Transformer to achieve linear complexity self-attention;
Lookahead When It Matters: Adaptive Non-causal Transformers for Streaming Neural Transducers
Grant Strimel, Athanasios Mouchtaris (Amazon)
RecognitionTransformerAudio
🎯 What it does: Proposes the Adaptive Non-Causal Attention Transducer (ANCAT), which dynamically determines how much future context to use for each frame in streaming speech recognition, balancing low latency and high accuracy.
LookupFFN: Making Transformers Compute-lite for CPU inference
Zhanpeng Zeng (University of Wisconsin), Vikas Singh (NVIDIA Research)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This study investigates a method to rewrite the Feed-Forward network in Transformer as a learnable memory lookup approach (LookupFFN), significantly reducing FLOP and making it more suitable for CPU inference.
Looped Transformers as Programmable Computers
Angeliki Giannou (University of Wisconsin-Madison), Dimitris Papailiopoulos (University of Wisconsin-Madison)
TransformerSequential
🎯 What it does: A cyclic Transformer framework has been constructed, allowing a preset-weight Transformer to execute general computational tasks according to instruction sequences, thereby simulating a programmable computer;
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Yixiao Li (Georgia Institute of Technology), Tuo Zhao (Georgia Institute of Technology)
CompressionComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelTextMultimodality
🎯 What it does: This paper proposes a new structured compression method called LoSparse, which approximates the Transformer weight matrix using a decomposition of low-rank and sparse matrices, significantly reducing the number of parameters and computational cost while maintaining model performance.
Loss Balancing for Fair Supervised Learning
Mohammad Mahdi Khalili (Yahoo Research), Mahed Abroshan (Optum Labs)
OptimizationTabular
🎯 What it does: This paper proposes a new fair learning method that addresses the 'Equalized Loss' (EL) fairness constraint, achieving a balance of expected loss across different groups in supervised learning.
Loss-Guided Diffusion Models for Plug-and-Play Controllable Generation
Jiaming Song (NVIDIA Corporation), Arash Vahdat (NVIDIA Corporation)
GenerationSuper ResolutionDiffusion modelImageText
🎯 What it does: Proposes the Loss-Guided Diffusion (LGD) framework, achieving plug-and-play control generation of diffusion models through any differentiable loss function without training.
Lottery Tickets in Evolutionary Optimization: On Sparse Backpropagation-Free Trainability
Robert Tjarko Lange (Technical University Berlin), Henning Sprekeler (Technical University Berlin)
OptimizationReinforcement LearningSequential
🎯 What it does: This paper studies the lottery ticket phenomenon in evolutionary optimization, proving the existence of trainable highly sparse network initialization in evolutionary strategies (ES) and proposing an iterative pruning method based on signal-to-noise ratio (SNR);
Low Complexity Homeomorphic Projection to Ensure Neural-Network Solution Feasibility for Optimization over (Non-)Convex Set
Enming Liang (City University of Hong Kong), Steven Low
OptimizationFlow-based ModelTabular
🎯 What it does: A low-complexity 'Homeomorphic Projection' (HP) framework is proposed, which utilizes reversible neural networks to learn the minimum distortion homeomorphic mapping between the constraint set and the unit sphere. It then recovers the feasible solution to the original problem in the unit sphere space through a bisection method, ensuring that the neural network's predicted solution satisfies all constraints.
Low-Switching Policy Gradient with Exploration via Online Sensitivity Sampling
Yunfan Li (University of California), Lin Yang (University of California)
OptimizationReinforcement LearningSequential
🎯 What it does: A low-switching, sample-efficient policy gradient optimization algorithm LPO is proposed, supporting nonlinear function approximation.
Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single
Paul Vicol (Google Brain)
OptimizationHyperparameter SearchRecurrent Neural NetworkReinforcement LearningSequential
🎯 What it does: A new gradient estimation method based on evolutionary strategies, ES-Single, was designed and evaluated, which can estimate external parameter gradients in a low-variance and unbiased manner within truncated unfolded computation graphs.
Lower Bounds for Learning in Revealing POMDPs
Fan Chen (Peking University), Yu Bai (Salesforce AI Research)
Reinforcement Learning
🎯 What it does: This paper constructs difficult instances to provide a lower bound on the optimal sample complexity of revealing POMDPs in PAC learning and regret learning, and proves that these lower bounds are close to known upper bounds, indicating that revealing POMDPs are sample-efficient for learning.
Lowering the Pre-training Tax for Gradient-based Subset Training: A Lightweight Distributed Pre-Training Toolkit
Yeonju Ro (University of Texas at Austin), Aditya Akella (University of Texas at Austin)
OptimizationComputational EfficiencyConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A distributed, communication-free lightweight pre-training framework is proposed, aimed at significantly reducing the pre-training cost in gradient subset training and thereby improving the final performance of subset training.
LSDS++ : Dual Sampling for Accelerated k-means++
Chenglin Fan (Pennsylvania State University), Xiaoyun Li (LinkedIn)
OptimizationComputational EfficiencyBiomedical Data
🎯 What it does: This paper proposes an accelerated version of k-means++ called LSDS++, which is based on dual sampling. It achieves local search with a computational complexity of O(nd) at each step by only swapping randomly selected candidate points with the nearest center and a random center, achieving a constant expected error equivalent to LocalSearch++.
MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior
Jennifer J. Sun (California Institute of Technology), Ann Kennedy (Northwestern University)
ClassificationRecognitionRepresentation LearningContrastive LearningVideoMultimodalityBenchmark
🎯 What it does: A cross-species multi-task benchmark MABe22 has been constructed to evaluate self-supervised representation learning of animal interaction behaviors.
Machine Learning Force Fields with Data Cost Aware Training
Alexander Bukharin (Georgia Institute of Technology), Tuo Zhao (ByteDance Inc)
Graph Neural NetworkScore-based ModelTabularPhysics Related
🎯 What it does: The ASTEROID multi-stage training framework is proposed, which significantly reduces the cost of training data by first learning the force field structure using a large amount of low-cost inaccurate data (such as DFT or empirical force fields) and then fine-tuning with a small amount of high-precision data (such as CCSD(T)).
MAGANet: Achieving Combinatorial Generalization by Modeling a Group Action
Geonho Hwang (Korea Institute for Advanced Study), Myungjoo Kang (Seoul National University)
GenerationData SynthesisAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes MAGANet, a generative framework that learns data transformations by modeling group actions, aimed at achieving compositional generalization.
Magneto: A Foundation Transformer
Hongyu Wang (University of Chinese Academy of Sciences), Furu Wei (Microsoft)
TransformerImageTextMultimodalityAudio
🎯 What it does: A general Transformer architecture named MAGNETO is proposed, which achieves better expressiveness and training stability by adding an extra LayerNorm (Sub‑LayerNorm) to each sub-layer and implementing initialization derived from deep network theory.
MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations
Anqi Li (University of Washington), Ching-An Cheng (Microsoft Research)
Reinforcement Learning
🎯 What it does: A new paradigm of offline decision learning is proposed - Policy Learning from Observations (PLfO), which unifies offline reinforcement learning and imitation learning from observations.
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models
Rongjie Huang (Zhejiang University), Zhou Zhao (Zhejiang University)
GenerationData SynthesisPrompt EngineeringDiffusion modelMultimodalityAudio
🎯 What it does: A text-to-audio (T2A) generation framework called Make‑An‑Audio is proposed, which achieves high-quality audio synthesis by combining a pseudo-prompt enhanced diffusion model.
MANSA: Learning Fast and Slow in Multi-Agent Systems
David Henry Mguni (Huawei Research and Development), Yaodong Yang (Peking University)
Reinforcement Learning
🎯 What it does: This paper proposes the MANSA framework, which can dynamically switch between Independent Learning (IL) and Centralized Learning (CL) in multi-agent reinforcement learning based on the environmental state, significantly reducing the number of CL calls while maintaining convergence.
Margin-based Neural Network Watermarking
Byungjoo Kim (KAIST), Sung Ju Hwang (KAIST)
ClassificationOptimizationKnowledge DistillationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A DNN watermarking method based on maximizing the margin of trigger set samples is proposed, which can maintain a high watermark recognition rate even under black-box model extraction and distillation attacks.
Margin-based sampling in high dimensions: When being active is less efficient than staying passive
Alexandru Tifrea (ETH Zurich), Fanny Yang (ETH Zurich)
OptimizationTabularFinance Related
🎯 What it does: This paper systematically studies the comparison between margin-based active learning (M-AL) and passive learning (PL) in high-dimensional environments from both theoretical and experimental perspectives. It proves and confirms that in scenarios with high dimensions and low annotation budgets, M-AL is not only computationally more expensive but often performs worse than PL; this conclusion is verified through a large number of high-dimensional datasets.
Marginalization is not Marginal: No Bad VAE Local Minima when Learning Optimal Sparse Representations
David Wipf (Amazon Web Services)
Representation LearningAuto EncoderImage
🎯 What it does: This paper theoretically analyzes the energy function of Variational Autoencoders (VAE) and proves that under certain conditions, VAE does not have undesirable local minima, and all local minima can achieve optimal sparse representations, comparing it with deterministic autoencoders (AE) of equal capacity.
Markovian Gaussian Process Variational Autoencoders
Harrison Zhu (Imperial College London), Yingzhen Li (Imperial College London)
GenerationData SynthesisComputational EfficiencyRecurrent Neural NetworkAuto EncoderVideoTime SeriesSequentialStochastic Differential Equation
🎯 What it does: A variational autoencoder based on Markov Gaussian processes (MGPVAE) is proposed, achieving O(T) time Kalman filtering and smoothing by mapping GP to a linear state space model.
Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inference
Insung Kong (Seoul National University), Yongdai Kim (Seoul National University)
ImageTabularTime Series
🎯 What it does: A node-sparse Bayesian neural network (mBNN) is proposed, proving that its posterior convergence rate is approximately optimal and adaptively smooth, along with designing a feasible MCMC inference method.
Masked Trajectory Models for Prediction, Representation, and Control
Philipp Wu (University of California Berkeley), Aravind Rajeswaran (Meta AI)
Robotic IntelligenceTransformerReinforcement LearningSequential
🎯 What it does: A Masked Trajectory Model (MTM) is proposed, which achieves general modeling of continuous decision-making tasks through self-supervised masked reconstruction training on trajectory sequences.
Master-ASR: Achieving Multilingual Scalability and Low-Resource Adaptation in ASR with Modular Learning
Zhongzhi Yu (Georgia Institute of Technology), Celine Lin
RecognitionDomain AdaptationTransformerSupervised Fine-TuningAudio
🎯 What it does: The Master-ASR framework is proposed, achieving scalability and low-resource adaptation for multilingual ASR through a modular reassembly of the Artisan Layer.
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
Sai Rajeswar (Mila), Alexandre Lacoste
Robotic IntelligenceReinforcement LearningWorld ModelImageBenchmark
🎯 What it does: This study proposes an unsupervised reinforcement learning pre-training method based on a world model, achieving efficient adaptation to visual control tasks under task-aware fine-tuning and mixed planning (Dyna-MPC).
Matrix Estimation for Individual Fairness
Cindy Zhang, Devavrat Shah (Massachusetts Institute of Technology)
Recommendation SystemSupervised Fine-TuningTabular
🎯 What it does: This paper proposes using Singular Value Thresholding (SVT) to preprocess sparse noisy data before model training, in order to enhance the algorithm's Individual Fairness (IF) without significantly sacrificing predictive performance.
Maximal Initial Learning Rates in Deep ReLU Networks
Gaurav Iyer (McGill University), David Rolnick (Mila - Quebec AI Institute)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper defines and studies the 'maximum initial learning rate (η∗)', which is the maximum learning rate at which a network can successfully start training and reach a specified threshold accuracy after random initialization. It is experimentally and theoretically proven that there is a power-law relationship between η∗ and the network architecture (depth × width) as well as the sharpness at initialization (λ1).
Maximum Optimality Margin: A Unified Approach for Contextual Linear Programming and Inverse Linear Programming
Chunlin Sun (Stanford University), Xiaocheng Li (Imperial College London)
OptimizationTabular
🎯 What it does: This paper studies the prediction-optimization problem, where the output of a machine learning prediction task is used as input for a downstream optimization problem. A new method called Maximum Optimality Margin is proposed to address the shortcomings of existing methods in terms of optimization infeasibility and statistical efficiency.
Measuring the Impact of Programming Language Distribution
Gabriel Orlanski (New York University), Michele Catasta (Google)
AI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The BabelCode framework is proposed to achieve multi-language executable evaluation, and a new translation dataset TP3 is built based on it; the Unimax strategy is used to balance the training distribution of 14 languages, training decoder-only models of 1B/2B/4B parameters.
Mechanistic Mode Connectivity
Ekdeep Singh Lubana (University of Michigan), Hidenori Tanaka (Harvard University)
ClassificationOptimizationAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This study investigates the relationship between pattern connectivity and model mechanism similarity in the loss landscape of neural networks, defines mechanism similarity (shared invariance), proves that a lack of linear connectivity implies dissimilar mechanisms, and proposes a connectivity-based fine-tuning method (CBFT) to efficiently suppress the model's dependence on shortcut attributes with few samples.
Memory-Based Dual Gaussian Processes for Sequential Learning
Paul Edmund Chang, Mohammad Emtiyaz Khan (RIKEN Center for AI Project)
Sequential
🎯 What it does: A memory-based dual sparse variational Gaussian process is proposed for sequential learning.
Memory-Based Meta-Learning on Non-Stationary Distributions
Tim Genewein (DeepMind), Joel Veness (DeepMind)
Meta LearningRecurrent Neural NetworkTransformerTime SeriesSequential
🎯 What it does: This study explores whether neural models trained using Memory-Based Meta-Learning (MBML) (such as LSTM, Transformer, RNN) can approximate the Bayesian optimal predictor under non-stationary distributions, focusing on piecewise Bernoulli sources with unknown switching points.
Men Also Do Laundry: Multi-Attribute Bias Amplification
Dora Zhao (Sony AI), Alice Xiang (Sony AI)
Convolutional Neural NetworkImage
🎯 What it does: This paper proposes a Multi-Attribute Bias Amplification measurement method and experimentally validates that this measurement can reveal bias amplification phenomena overlooked by traditional single-attribute measurements; it also evaluates the effectiveness of various bias mitigation methods under this measurement.
Meta Learning of Interface Conditions for Multi-Domain Physics-Informed Neural Networks
Shibo Li (University of Utah), Shandian Zhe (University of Utah)
OptimizationMeta LearningTabularPhysics Related
🎯 What it does: This paper proposes the METALIC method, which utilizes contextual multi-armed bandits (MAB) to dynamically learn the interface conditions of multi-domain PINNs, thereby improving the accuracy of PDE solutions.
Meta Optimal Transport
Brandon Amos (Meta), Ievgen Redko (Huawei)
OptimizationMeta LearningImage
🎯 What it does: Using meta-learning/adaptive optimization techniques, a model is trained to directly predict the parameters of the dual potential or continuous potential for optimal transport given a pair of measures (α, β) and cost c, thereby obtaining a warm-start transport plan that significantly accelerates subsequent Sinkhorn or W2GN solving.
Meta-learning Parameterized Skills
Haotian Fu (Brown University), George Konidaris (Brown University)
Meta LearningReinforcement LearningSequential
🎯 What it does: This paper proposes a parameterized skill learning framework based on offline Meta-RL and trajectory smoothing regularization, and builds a three-layer hierarchical RL system on this basis to address long-horizon tasks.
Meta-Learning the Inductive Bias of Simple Neural Circuits
Will Dorrell, Peter E. Latham
Meta LearningSpiking Neural NetworkImageGraph
🎯 What it does: This paper proposes a meta-learning-based framework that allows an outer meta-learner to assign labels to samples, under which an inner learner is trained. The meta-learner then updates through gradients to minimize the generalization error of the learner on unseen data, thereby extracting the learner's inductive bias (i.e., functions that are easy to generalize).
Meta-SAGE: Scale Meta-Learning Scheduled Adaptation with Guided Exploration for Mitigating Scale Shift on Combinatorial Optimization
Jiwoo Son (Korea Advanced Institute of Science and Technology), Jinkyoo Park (Korea Advanced Institute of Science and Technology)
OptimizationMeta LearningReinforcement LearningTabular
🎯 What it does: The Meta-SAGE method enables rapid adaptation of pre-trained reinforcement learning construction models to large-scale combinatorial optimization problems.
MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL
Fei Ni (Tianjin University), Zhixuan Liang (Hong Kong University)
Meta LearningReinforcement LearningDiffusion modelSequential
🎯 What it does: This paper proposes MetaDiffuser, an offline meta reinforcement learning framework based on conditional diffusion models, which can quickly adapt to unseen tasks given a small amount of warm-start data.
Metagenomic Binning using Connectivity-constrained Variational Autoencoders
Andre Lamurias (Aalborg University), Thomas Dyhre Nielsen (Aalborg University)
Auto EncoderGraph
🎯 What it does: This study proposes a metagenomic binning method based on the Connected Constraint Variational Autoencoder (CCVAE), which utilizes the connectivity information of the assembly graph and single-copy gene (SCG) features to constrain the latent representation of contigs, thereby improving binning quality.
MetaModulation: Learning Variational Feature Hierarchies for Few-Shot Learning with Fewer Tasks
Wenfang Sun (Hefei Institutes of Physical Science Chinese Academy of Sciences), Cees G. M. Snoek (University of Amsterdam)
Meta LearningImageTabular
🎯 What it does: Proposes the MetaModulation method, which modulates multi-layer batch normalization parameters using task conditions to achieve few-task meta-learning;
MetricGAN-OKD: Multi-Metric Optimization of MetricGAN via Online Knowledge Distillation for Speech Enhancement
Wooseok Shin (Korea University), Sung Won Han (Korea University)
GenerationOptimizationKnowledge DistillationGenerative Adversarial NetworkAudio
🎯 What it does: This paper proposes an improved architecture of MetricGAN called MetricGAN-OKD, achieved through online knowledge distillation, for speech enhancement and auditory enhancement tasks.
MEWL: Few-shot multimodal word learning with referential uncertainty
Guangyuan Jiang (Peking University), Yixin Zhu (Peking University)
TransformerLarge Language ModelPrompt EngineeringMultimodalityBenchmark
🎯 What it does: Proposes the MEWL benchmark to evaluate the ability of machines to learn word meanings across contexts with reference uncertainty under limited examples.
MG-GNN: Multigrid Graph Neural Networks for Learning Multilevel Domain Decomposition Methods
Ali Taghibakhshi (University of Illinois at Urbana-Champaign), Matthew West (University of Illinois at Urbana-Champaign)
OptimizationGraph Neural NetworkGraph
🎯 What it does: A multi-level graph neural network (MG-GNN) is proposed to learn a two-layer optimized Restricted Additive Schwarz (ORAS) preconditioner, capable of simultaneously optimizing subdomain Robin boundary conditions and coarse grid interpolation operators.
Mimetic Initialization of Self-Attention Layers
Asher Trockman (Carnegie Mellon University), J Zico Kolter
TransformerImageText
🎯 What it does: This paper proposes a no-learning, closed initialization method based on pre-trained Transformer weight features, which ensures that the query-key matrix product of the self-attention layer approximates the identity matrix, and the value-projection matrix product approximates the negative identity matrix, thereby achieving a 'mimicking' effect of pre-training.
Minimalistic Predictions to Schedule Jobs with Online Precedence Constraints
Alexandra Lassota (Institute of Mathematics, EPFL), Jens Schlöter (University of Bremen)
Optimization
🎯 What it does: This paper studies the non-clairvoyant scheduling problem with online predecessor constraints, proposing various prediction models and designing algorithms within a learning-enhanced framework, achieving better competitive ratios than the traditional worst-case Ω(n).
Minimax estimation of discontinuous optimal transport maps: The semi-discrete case
Aram-Alexandre Pooladian (New York University), Jonathan Niles-Weed (New York University)
OptimizationTabular
🎯 What it does: The study estimates the statistical properties of discontinuous optimal transport mappings in the semi-discrete case and provides the optimal estimator.
Minimizing Trajectory Curvature of ODE-based Generative Models
Sangyun Lee (Soongsil University), Jong Chul Ye (KAIST)
GenerationData SynthesisOptimizationDiffusion modelRectified FlowImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This study investigates the trajectory curvature of ODE/SDE generative models and proposes a method to reduce curvature by learning the forward process, enabling faster and more accurate sampling.
Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation
Li'ang Li (Beijing Normal University), Yongqiang Cai (Beijing Normal University)
Ordinary Differential Equation
🎯 What it does: This paper studies the uniform approximation of leaky-ReLU neural networks in continuous function spaces, providing the exact value for their minimum width. It proves that when the output dimension equals the input dimension plus one, the width must be d_x + 1; in other cases, the width is max(d_x + 1, d_y).
Mirror Sinkhorn: Fast Online Optimization on Transport Polytopes
Marin Ballu (University of Cambridge), Quentin Berthet (Google DeepMind)
OptimizationImagePoint Cloud
🎯 What it does: A single-loop algorithm named Mirror Sinkhorn is proposed for minimizing arbitrary convex objectives on the transport polytope, supporting online learning and stochastic gradient updates.
Mitigating Memorization of Noisy Labels by Clipping the Model Prediction
Hongxin Wei (Southern University of Science and Technology), Yixuan Li (University of Wisconsin-Madison)
ClassificationConvolutional Neural NetworkImage
🎯 What it does: Proposes the LogitClip method, which limits the upper bound of loss functions such as cross-entropy by truncating the norm of the logit vector, thereby reducing overfitting caused by noisy labels.
Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling
Arka Daw (Virginia Tech), Anuj Karpatne (Virginia Tech)
OptimizationPhysics Related
🎯 What it does: A Retain-Resample-Release (R3) sampling algorithm is proposed to alleviate the propagation failure problem in Physics-Informed Neural Networks (PINN), and its effectiveness is validated.
Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning
Yu Yang (University of California), Baharan Mirzasoleiman (University of California)
Object DetectionSegmentationData-Centric LearningTransformerSupervised Fine-TuningVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: During the fine-tuning phase of multimodal models (such as CLIP), a language expression-based contrastive loss method is proposed to detect and explicitly remove spurious attributes from the training data, primarily achieved by adding additional spurious contrastive loss at the projection layer.
MixFlows: principled variational inference via mixed flows
Zuheng Xu (University of British Columbia), Trevor Campbell (University of British Columbia)
Flow-based ModelTabular
🎯 What it does: A new variational family called Mixed Variational Flows (MixFlows) is proposed, which averages the reference distribution using multiple pushforward mappings to achieve a sampleable and evaluable approximate posterior;
Mixing Predictions for Online Metric Algorithms
Antonios Antoniadis (University of Twente), Bertrand Simon (CNRS IN2P3 Computing Center)
Optimization
🎯 What it does: Designed and analyzed a class of learning-enhanced online metric algorithms that can mix multiple predictions, providing competitive guarantees under given predictions.
Mixture Proportion Estimation Beyond Irreducibility
Yilun Zhu (University of Michigan), Clayton Scott (University of Michigan)
OptimizationReinforcement LearningTabularSequentialBiomedical Data
🎯 What it does: A more relaxed identifiable condition than the traditional irreducibility assumption is proposed, and based on this condition, a general sampling subsampling (SuMPE) meta-algorithm is designed to estimate the mixing ratio.
Moccasin: Efficient Tensor Rematerialization for Neural Networks
Burak Bartan (Qualcomm Technologies), Bistra Dilkina (University of Southern California)
OptimizationComputational EfficiencyGraph
🎯 What it does: A Tensor rematerialization method based on constraint programming (MOCCASIN) is proposed, which minimizes the total execution time of the neural network computation graph while satisfying memory budget constraints by retaining interval models.