ICLR 2023 Papers with AI Summaries
International Conference on Learning Representations · 1573 papers
→ ICLR 2023 papers with code (737)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 1573 ICLR 2023 papers by keyword, author or institution —
free trial on arXivSub.
BEEF: Bi-Compatible Class-Incremental Learning via Energy-Based Expansion and Fusion
Fu-Yun Wang (Nanjing University), Peilin Zhao (Tencent AI Lab)
ClassificationContrastive LearningImage
🎯 What it does: The BEEF framework is proposed to address catastrophic forgetting in class-incremental learning, using an energy-based method for independent training and fusion of modules, while considering the compatibility of new and old tasks.
Cycle-consistent Masked AutoEncoder for Unsupervised Domain Generalization
Haiyang Yang (Nanjing University), Wanli Ouyang (Shanghai AI Laboratory)
Domain AdaptationRepresentation LearningTransformerAuto EncoderContrastive LearningImage
🎯 What it does: Proposes Cycle-consistent Masked AutoEncoder (CycleMAE) for unsupervised domain generalization, designing a self-cyclic cross-domain reconstruction task and incorporating domain contrast loss to help the model learn domain-invariant features.
Decompose to Generalize: Species-Generalized Animal Pose Estimation
Guangrui Li (University of Technology Sydney), Yi Yang (Zhejiang University)
Pose EstimationConvolutional Neural NetworkImage
🎯 What it does: This paper addresses the cross-species generalization problem in animal pose estimation by proposing to enhance the model's performance on unseen species through the decomposition of joint relationships.
(Certified!!) Adversarial Robustness for Free!
Nicholas Carlini (Google), J Zico Kolter
ClassificationAdversarial AttackTransformerDiffusion modelImage
🎯 What it does: This paper proposes a no-training 'denoised random smoothing' method that combines pre-trained diffusion models (Diffusion Probabilistic Models) with existing classifiers to achieve provable robustness of images based on the L2 norm.
$\Lambda$-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells
Sajad Movahedi (University of Tehran), Babak N Araabi (University of Tehran)
OptimizationNeural Architecture SearchImage
🎯 What it does: In the gradient-based differentiable architecture search method DARTS, the authors analyze the convergence issues caused by the weight-sharing framework and propose the 'Layer Alignment (Λ)' metric. They design two regularization terms to enhance the consistency of layer gradients, thereby alleviating the performance collapse problem, and introduce the Λ-DARTS method.
$\mathcal{O}$-GNN: incorporating ring priors into molecular modeling
Jinhua Zhu (University of Science and Technology of China), Tie-Yan Liu (Microsoft Research AI4Science)
Drug DiscoveryGraph Neural NetworkGraph
🎯 What it does: A ring-enhanced graph neural network (O-GNN) is proposed, explicitly modeling molecular rings and iteratively updating them along with atoms and bonds.
$\mathrm{SE}(3)$-Equivariant Attention Networks for Shape Reconstruction in Function Space
Evangelos Chatzipantazis (University of Pennsylvania), Kostas Daniilidis (University of Pennsylvania)
Object DetectionGenerationTransformerPoint Cloud
🎯 What it does: This paper proposes a SE(3)-equivariant attention network, TF-Onet, for reconstructing 3D shapes from sparse, unstructured point clouds.
$\mathscr{N}$-WL: A New Hierarchy of Expressivity for Graph Neural Networks
Qing Wang (Australian National University), Muhammad Farhan (Australian National University)
ClassificationRecognitionGraph Neural NetworkGraph
🎯 What it does: A neighborhood subgraph-based Weisfeiler-Leman hierarchy (N-WL) is proposed to measure the expressiveness of graph neural networks, and based on this, the Graph Neighbourhood Neural Network (G3N) model is designed;
$\rm A^2Q$: Aggregation-Aware Quantization for Graph Neural Networks
Zeyu Zhu (Institute of Automation, Chinese Academy of Sciences), Jian Cheng (Institute of Automation, Chinese Academy of Sciences)
CompressionComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: A mixed-precision quantization method called A Q2 is proposed, which compresses the model size and computational load of graph neural networks (GNNs) while maintaining almost no loss in accuracy, thereby improving inference speed.
$k$NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference
Benfeng Xu (University of Science and Technology of China), Yongdong Zhang (Institute of Artificial Intelligence)
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A kNN-based prompting method called k NN Prompting is proposed, which utilizes the complete language model distribution generated by LLM as representation, constructs a local data storage, and makes predictions through nearest neighbor matching;
$O(T^{-1})$ Convergence of Optimistic-Follow-the-Regularized-Leader in Two-Player Zero-Sum Markov Games
Yuepeng Yang (University of Chicago), Cong Ma (University of Chicago)
OptimizationReinforcement Learning
🎯 What it does: An algorithm based on optimistic-follow-the-regularized-leader (OFTRL) combined with smooth value updates is proposed, achieving a Nash equilibrium approximation convergence rate of O(1/T) in finite time two-player zero-sum Markov games.
3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction
Jiaqi Guan (University of Illinois Urbana-Champaign), Jianzhu Ma (Tsinghua University)
GenerationDrug DiscoveryGraph Neural NetworkDiffusion modelGraph
🎯 What it does: Designed and implemented TargetDiff, a SE(3)-equivariant diffusion model for generating 3D molecules that match protein targets, which can be used for affinity ranking and prediction.
3D generation on ImageNet
Ivan Skorokhodov (King Abdullah University of Science and Technology), Sergey Tulyakov (Snap Inc.)
GenerationData SynthesisDepth EstimationKnowledge DistillationNeural Radiance FieldGenerative Adversarial NetworkImage
🎯 What it does: This paper presents 3DGP, a 3D-aware image generator that can be trained on non-aligned, multi-class datasets (such as ImageNet) using a universal deep prior and a learnable camera distribution.
3D Segmenter: 3D Transformer based Semantic Segmentation via 2D Panoramic Distillation
ZHENNAN WU, Hiroyuki Sato (University of Tokyo)
SegmentationKnowledge DistillationTransformerPoint Cloud
🎯 What it does: A 2D→3D knowledge distillation method based on panoramic images is proposed, utilizing a pre-trained 2D segmentation network to enhance 3D voxel semantic segmentation, and a 3D Segmenter network is introduced.
3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation
Ho Hin Lee (Vanderbilt University), Bennett A. Landman (Vanderbilt University)
SegmentationConvolutional Neural NetworkTransformerBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: This paper proposes 3D UX-Net, a lightweight 3D convolutional network that uses large kernel depthwise separable convolutions to simulate hierarchical Transformers for medical image segmentation.
A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification
Paul F Jaeger, Till J. Bungert (German Cancer Research Center)
ClassificationDomain AdaptationAnomaly DetectionConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a unified evaluation framework to compare different failure detection methods, revealing three major flaws in current evaluations, and validates through large-scale experiments that softmax response remains the best baseline.
A Closer Look at Model Adaptation using Feature Distortion and Simplicity Bias
Puja Trivedi (University of Michigan), Jayaraman J. Thiagarajan (Lawrence Livermore National Laboratory)
Domain AdaptationAnomaly DetectionConvolutional Neural NetworkSupervised Fine-TuningContrastive LearningImage
🎯 What it does: This paper studies how to enhance security and generalization performance while controlling feature distortion and reducing simplicity bias during adaptation on pre-trained models.
A CMDP-within-online framework for Meta-Safe Reinforcement Learning
Vanshaj Khattar (Virginia Tech), Ming Jin (Virginia Tech)
Meta LearningReinforcement LearningSequential
🎯 What it does: A CMDP-within-online framework is proposed, achieving provable low return and constraint violation upper bounds for reward and constraint violations in meta-safe reinforcement learning.
A Control-Centric Benchmark for Video Prediction
Stephen Tian (Stanford University), Jiajun Wu (Stanford University)
Robotic IntelligenceVideoBenchmark
🎯 What it does: A video prediction benchmark VP 2 aimed at control is proposed, and various action-conditioned video prediction models are evaluated and compared in terms of their performance in robotic visual planning using this benchmark.
A Convergent Single-Loop Algorithm for Relaxation of Gromov-Wasserstein in Graph Data
Jiajin Li (Stanford University), Jose Blanchet (Stanford University)
OptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: A single-loop Bregman Alternating Projection Gradient (BAPG) algorithm is proposed for approximately solving the Gromov-Wasserstein distance, balancing computational efficiency and matching accuracy.
A critical look at the evaluation of GNNs under heterophily: Are we really making progress?
Oleg Platonov (Higher School of Economics University), Liudmila Prokhorenkova (Yandex Research)
Graph Neural NetworkGraphBenchmark
🎯 What it does: Evaluate the performance of graph neural networks on heterogeneous graphs, revealing information leakage in traditional evaluation datasets and proposing a new benchmark for heterogeneous graphs.
A Differential Geometric View and Explainability of GNN on Evolving Graphs
Yazheng Liu (Beijing University of Posts and Telecommunications), Sihong Xie (Lehigh University)
OptimizationExplainability and InterpretabilityGraph Neural NetworkGraph
🎯 What it does: Proposes a smooth path contribution parameterization based on differential geometry, describing the evolution of probability distributions of GNNs in dynamic graphs, and explains the prediction changes accordingly.
A framework for benchmarking Class-out-of-distribution detection and its application to ImageNet
Ido Galil (Technion), Ran El-Yaniv (Technion)
Anomaly DetectionKnowledge DistillationTransformerImageBenchmark
🎯 What it does: A benchmark framework for out-of-distribution (C-OOD) detection with adjustable difficulty levels based on the model's own confidence is proposed, and an evaluation is conducted on the ImageNet-1k classifier.
A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis
Damien Ferbach (Ecole Normale Superieure PSL), Joey Bose
ClassificationGraph Neural NetworkImageGraph
🎯 What it does: This paper proposes a unified framework that proves that in sufficiently over-parameterized group equivariant networks, it is always possible to obtain a 'strong lottery ticket' that can achieve the target network accuracy at initialization through pruning.
A General Framework for Sample-Efficient Function Approximation in Reinforcement Learning
Zixiang Chen (University of California), Michael Jordan
OptimizationReinforcement Learning
🎯 What it does: A unified sampling efficiency function approximation framework (ABC) and the corresponding optimization sampling algorithm OPERA are proposed for achieving theoretically provable sample-efficient learning in a wide range of MDP models (including linear mixtures, low Witness Rank, KNR, etc.).
A General Rank Preserving Framework for Asymmetric Image Retrieval
Hui Wu (University of Science and Technology of China), Houqiang Li (University of Science and Technology of China)
RetrievalImage
🎯 What it does: A novel unsupervised generalization ranking preservation framework is proposed for deploying lightweight models on resource-constrained endpoints while using powerful models for asymmetric image retrieval in the cloud.
A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming
Qingyu Han (Shandong University), Xiaodong Luo (Shenzhen Research Institute of Big Data)
OptimizationGraph Neural NetworkTabular
🎯 What it does: This paper proposes a prediction-search framework based on graph neural networks (GNN) to quickly find high-quality feasible solutions for mixed-integer linear programming (MILP).
A Graph Neural Network Approach to Automated Model Building in Cryo-EM Maps
Kiarash Jamali (MRC Laboratory of Molecular Biology), Sjors HW Scheres
Protein Structure PredictionGraph Neural NetworkLarge Language ModelImage
🎯 What it does: Automated protein chain modeling and atomic coordinate prediction were achieved through graph neural networks, capable of directly generating atomic models close to those constructed manually from cryo-EM 3D density maps.
A Higher Precision Algorithm for Computing the $1$-Wasserstein Distance
Pankaj K Agarwal, Rachita Sowle (Virginia Tech)
OptimizationTabular
🎯 What it does: A new accelerated high-precision approximation algorithm is designed for the discrete 1-Wasserstein distance (and the corresponding Euclidean Bipartite Matching problem) between two distributions in a d-dimensional unit hypercube. This algorithm combines a hierarchical quadtree structure with any additive approximation algorithm to achieve a relative accuracy improvement beyond the original additive error ε.
A Holistic View of Label Noise Transition Matrix in Deep Learning and Beyond
LIN Yong, Bo Han (Hong Kong Baptist University)
ClassificationOptimizationImage
🎯 What it does: This paper proposes a learning framework for statistically consistent classifiers in the presence of label noise by estimating the noise transition matrix (T). It first summarizes existing T estimation methods as the Minimum Geometric Envelope Operator (MGEO) and points out its inconsistency under posterior estimation errors. Subsequently, it introduces the Robust Bilevel Optimization (ROBOT) framework, which utilizes bilevel optimization and robust loss to achieve identifiable, consistent, and finite sample generalization guarantees for T without requiring perfect posterior estimation and anchor point assumptions.
A Kernel Perspective of Skip Connections in Convolutional Networks
Daniel Barzilai (Weizmann Institute of Science), Ronen Basri (Weizmann Institute of Science)
Convolutional Neural Network
🎯 What it does: Derived closed recursive formulas for the Gaussian Process Kernel (CGPK) and Neural Tangent Kernel (CNTK) of convolutional residual networks, analyzed their spectra, and provided upper and lower bounds for eigenvalue decay, locality bias, and condition number.
A Laplace-inspired Distribution on SO(3) for Probabilistic Rotation Estimation
Yingda Yin (Peking University), Baoquan Chen (Peking University)
Pose EstimationConvolutional Neural NetworkPoint CloudBenchmark
🎯 What it does: Proposed the Rotation Laplace distribution for probabilistic rotation regression on SO(3) and realized its equivalent distribution for quantum rotations.
A law of adversarial risk, interpolation, and label noise
Daniel Paleka (ETH Zurich), Amartya Sanyal (ETH Zurich)
Adversarial AttackImage
🎯 What it does: The study investigates how label noise in overfitting models with labeled noise leads to an increase in adversarial risk.
A Learning Based Hypothesis Test for Harmful Covariate Shift
Tom Ginsberg (University of Toronto), Rahul G Krishnan
ClassificationDomain AdaptationAnomaly DetectionMultimodalityTabular
🎯 What it does: A method called Detectron is proposed, based on a Constraint Inconsistency Classifier (CDC) and ensemble learning, for the rapid detection of harmful covariate shifts during deployment.
A Message Passing Perspective on Learning Dynamics of Contrastive Learning
Yifei Wang (Peking University), Yisen Wang (Peking University)
Representation LearningGraph Neural NetworkTransformerContrastive LearningImage
🎯 What it does: This paper maps the contrastive learning loss to the feature space from an information theory perspective, re-decomposing it into two components: alignment and uniformity. It shows that the gradient descent of these two components is equivalent to information propagation on the augmentation graph and the affinity graph, providing an analytical description of the learning dynamics.
A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics
Qing Li (National Key Laboratory of General Artificial Intelligence), Song-Chun Zhu (Institute for Artificial Intelligence)
Convolutional Neural NetworkRecurrent Neural NetworkTransformerLarge Language ModelPrompt EngineeringImageTextBenchmark
🎯 What it does: A handwritten arithmetic dataset HINT is proposed to evaluate the system generalization ability of models in perception, syntax, and semantics, and benchmark experiments are conducted on existing seq2seq models.
A Mixture-of-Expert Approach to RL-based Dialogue Management
Yinlam Chow (Google Research), Craig Boutilier (Google Research)
TransformerReinforcement LearningMixture of ExpertsText
🎯 What it does: A reinforcement learning dialogue management system based on a mixture of experts (MoE) language model has been constructed, which can generate diverse, intent-driven candidate responses in the latent space of dialogue history and select the best response through reinforcement learning.
A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning
Da-Wei Zhou (Nanjing University), De-Chuan Zhan (Nanjing University)
ClassificationComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a memory-efficient class-incremental learning method called MEMO, and for the first time compares various CIL methods under a unified memory budget;
A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification
Xiang Hu (Ant Group), Kewei Tu (ShanghaiTech University)
ClassificationExplainability and InterpretabilityTransformerText
🎯 What it does: A Symbolic-Neural model is proposed, which learns single-label or multi-label classification of text through a structured language model (Structured LM) without span labels, and predicts span-level labels at internal nodes, forming an interpretable label tree.
A Neural Mean Embedding Approach for Back-door and Front-door Adjustment
Liyuan Xu (Gatsby Unit), Arthur Gretton (Gatsby Unit)
ImageTabular
🎯 What it does: This paper proposes a two-stage regression framework based on neural mean embedding for estimating causal effects in backdoor and frontdoor settings; in the first stage, conditional expectations are learned through neural networks, and in the second stage, the expected value is computed directly by obtaining feature means.
A new characterization of the edge of stability based on a sharpness measure aware of batch gradient distribution
Sungyoon Lee (Hanyang University), Cheongjae Jang (Hanyang University)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an Interaction-Aware Sharpness (IAS) metric based on batch gradient distribution, and uses it to reframe the 'Edge of Stability' (EoS) phenomenon in gradient descent and stochastic gradient descent. It further presents Implicit Interaction Regularization (IIR) and a new learning rate and batch size scaling rule (Linear and Saturation Scaling Rule, LSSR).
A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks
Xinyi Wu (Massachusetts Institute of Technology), Ali Jadbabaie (Massachusetts Institute of Technology)
Graph Neural NetworkGraph
🎯 What it does: A non-asymptotic quantitative analysis of the oversmoothing phenomenon in Graph Neural Networks (GNNs) is conducted, distinguishing between mixing effects and denoising effects, and providing an upper bound on the optimal number of layers O(log N / log log N).
A Non-monotonic Self-terminating Language Model
Eugene Choi (New York University), Cheolhyoung Lee (New York University)
GenerationRecurrent Neural NetworkTransformerLarge Language ModelText
🎯 What it does: This study investigates the issue of generating infinitely long sequences (non-terminating) when using incomplete probability decoding algorithms in autoregressive language models, and proposes a Non-Monotonic Self-Terminating (NMST) language model to address this problem.
A Primal-Dual Framework for Transformers and Neural Networks
Tan Minh Nguyen, Stanley Osher
ClassificationComputational EfficiencyTransformerImageTime SeriesBenchmark
🎯 What it does: This paper constructs a primal-dual framework for self-attention and neural network layers by treating self-attention as the support vector expansion of the support vector regression (SVR) problem. Within this framework, two novel attention mechanisms are proposed: Batch Normalized Attention (Attention-BN) and Attention with Scaled Heads (Attention-SH), along with their combination.
A probabilistic framework for task-aligned intra- and inter-area neural manifold estimation
Edoardo Balzani (New York University), Cristina Savin (New York University)
Time Series
🎯 What it does: A probabilistic framework TAME-GP based on Gaussian process priors is proposed, which can separate shared and private variations within and outside of neural populations in a single trial and align them with task variables.
A Self-Attention Ansatz for Ab-initio Quantum Chemistry
Ingrid von Glehn (DeepMind), David Pfau (DeepMind)
TransformerTabularPhysics Related
🎯 What it does: This paper proposes a self-attention-based wave function transformer (Psiformer) that can serve as an approximate Ansatz for the multi-electron Schrödinger equation.
A Simple Approach for Visual Room Rearrangement: 3D Mapping and Semantic Search
Brandon Trabucco (Carnegie Mellon University), Ruslan Salakhutdinov (Carnegie Mellon University)
Object DetectionSegmentationRobotic IntelligenceReinforcement LearningPoint Cloud
🎯 What it does: This paper proposes a concise visual room rearrangement framework that utilizes two voxel-based semantic maps (target state and current state) along with a semantic search strategy to first locate the objects to be rearranged, then infer the target positions through map differences, and finally plan movements to achieve the rearrangement.
A Simple Yet Powerful Deep Active Learning With Snapshots Ensembles
Seohyeon Jung (Korea Advanced Institute of Science and Technology), Juho Lee (Korea Advanced Institute of Science and Technology)
ClassificationOptimizationComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: This study focuses on deep active learning and proposes the use of Snapshot Ensembles to estimate uncertainty and build efficient active learning algorithms.
A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks
Marc Anton Finzi (New York University), Andrew Gordon Wilson (University of British Columbia)
Physics RelatedOrdinary Differential Equation
🎯 What it does: A local time method called Neural-IVP is proposed, which uses neural networks to approximate the solution of initial value PDEs, avoiding the catastrophic forgetting caused by traditional global minimization.
A Statistical Framework for Personalized Federated Learning and Estimation: Theory, Algorithms, and Privacy
Kaan Ozkara (University of California), Suhas Diggavi (University of California)
Federated LearningSafty and PrivacySupervised Fine-TuningTabular
🎯 What it does: A statistical framework based on empirical Bayes is proposed to unify and design personalized federated learning and estimation, providing privatized estimation and learning algorithms that satisfy communication and privacy constraints within this framework.
A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation
Hiroki Furuta (University of Tokyo), Shixiang Shane Gu (Google Research)
Knowledge DistillationRobotic IntelligenceReinforcement Learning from Human FeedbackGraph Neural NetworkTransformerReinforcement LearningAgentic AIGraphBenchmark
🎯 What it does: This paper proposes a unified Morphology-Task Graph (MTG) representation and implements offline behavior distillation for multiple morphologies and tasks in the MxT-Bench environment, thereby training a single policy that can generalize across various morphologies and tasks.
A Theoretical Framework for Inference and Learning in Predictive Coding Networks
Beren Millidge (University of Oxford), Rafal Bogacz (University of Oxford)
Image
🎯 What it does: This paper presents a theoretical framework for the Predictive Coding Network (PCN) when trained using prospective configuration, explaining the relationship between its inference equilibrium point and Target Propagation (TP) and Back Propagation (BP);
A theoretical study of inductive biases in contrastive learning
Jeff Z. HaoChen (Stanford University), Tengyu Ma (Stanford University)
OptimizationRepresentation LearningContrastive LearningImage
🎯 What it does: Theoretical analysis of inductive bias in contrastive learning is conducted, proving that an appropriate model architecture can reduce representation dimensions and improve downstream task performance.
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity
Hongkang Li (Rensselaer Polytechnic Institute), Pin-Yu Chen (IBM Research)
ClassificationTransformerImage
🎯 What it does: The paper conducts a theoretical analysis of a shallow Vision Transformer with a single layer of self-attention and two layers of perceptrons in a binary classification task, deriving the upper bound of sample complexity and proving that SGD can converge to zero generalization error under appropriate initialization.
A Theory of Dynamic Benchmarks
Ali Shirali (University of California), Moritz Hardt (Max-Planck Institute for Intelligent Systems)
ImageTextBenchmark
🎯 What it does: Researches the theoretical framework of dynamic benchmarks and conducts a rigorous analysis of common path-based dynamic benchmarks;
A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
Yuqi Nie (Princeton University), Jayant Kalagnanam (IBM Research)
OptimizationRepresentation LearningTransformerTime Series
🎯 What it does: This paper proposes PatchTST, a Transformer-based multivariate time series forecasting model that uses subsequence patches as input tokens and adopts a channel-independent design.
A Unified Algebraic Perspective on Lipschitz Neural Networks
Alexandre Araujo (INRIA), Bin Hu (University of Illinois Urbana-Champaign)
OptimizationAdversarial AttackImage
🎯 What it does: This paper presents a unified algebraic perspective aimed at designing and training neural networks with controlled Lipschitz constants to enhance robustness against adversarial attacks.
A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games
Samuel Sokota (Carnegie Mellon University), Christian Kroer (Columbia University)
OptimizationReinforcement LearningTabular
🎯 What it does: Proposes the Magnetic Mirror Descent (MMD) algorithm for solving quantum response equilibria (QRE) and reinforcement learning strategies in two-player zero-sum games, and proves its convergence in linear time.
A Unified Framework for Soft Threshold Pruning
Yanqi Chen (Peking University), Yonghong Tian (Peking University)
OptimizationConvolutional Neural NetworkSpiking Neural NetworkImage
🎯 What it does: This paper reinterprets soft threshold pruning as an implicit ISTA optimization problem and, based on this theoretical framework, designs learning rate adaptive threshold scheduling (LATS), simplified threshold scheduling (S-LATS), and continuation strategies (PGH scheduling), achieving efficient pruning during the sparsity increase process.
A VAE for Transformers with Nonparametric Variational Information Bottleneck
James Henderson (Idiap Research Institute), Fabio James Fehr
GenerationCompressionTransformerAuto EncoderText
🎯 What it does: A variational autoencoder for Transformers (NVAE) is proposed, which treats the multi-vector embeddings of Transformers as a variable-sized mixture distribution and uses non-parametric variational information bottleneck (NVIB) for regularization.
A View From Somewhere: Human-Centric Face Representations
Jerone Theodore Alexander Andrews, Alice Xiang (Sony AI)
ClassificationRecognitionRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This study constructed the AVFS dataset by collecting 638,180 triplet judgments of facial similarity and learned a facial embedding space that aligns with human perception.
A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta.
Maksim Velikanov (Technology Innovation Institute), Dmitry Yarotsky (Skolkovo Institute of Science and Technology)
OptimizationImageStochastic Differential Equation
🎯 What it does: An analytical framework based on Spectral Expressibility (SE) approximation and generating functions is proposed to study the noise-averaged convergence properties of batch SGD with constant learning rates and momentum under linear models.
AANG : Automating Auxiliary Learning
Lucio M. Dery (Carnegie Mellon University), Ameet Talwalkar (Hewlett Packard Enterprise)
Data-Centric LearningMeta LearningTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a framework for the automated generation and use of auxiliary objectives, and based on this, designs the AANG algorithm for task-aware multi-task learning on specific end tasks, thereby improving the model's performance on low-resource NLP tasks.
Accelerated Single-Call Methods for Constrained Min-Max Optimization
Yang Cai (Yale University), Weiqiang Zheng (Yale University)
OptimizationReinforcement LearningGenerative Adversarial Network
🎯 What it does: This paper studies first-order methods for constrained min-max optimization and proposes algorithms that require a single call and a single projection, addressing the issue of existing methods needing two gradient calls or two projections in each iteration.
Accelerating Guided Diffusion Sampling with Splitting Numerical Methods
Suttisak Wizadwongsa (Vistec), Supasorn Suwajanakorn (Vistec)
GenerationData SynthesisSuper ResolutionDiffusion modelImageOrdinary Differential Equation
🎯 What it does: This study investigates the reasons for the poor performance of high-order numerical methods in guided diffusion sampling and proposes a Strang splitting method based on operator splitting to significantly accelerate conditional diffusion sampling.
Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time
Jun-Kun Wang (Yale University), Andre Wibisono (Yale University)
OptimizationComputational EfficiencyTabularStochastic Differential Equation
🎯 What it does: This paper proposes a time-varying integration time scheme based on Chebyshev polynomials to accelerate the Hamiltonian Monte Carlo (HMC) method in the process of sampling from distributions.
Accurate Bayesian Meta-Learning by Accurate Task Posterior Inference
Michael Volpp (Karlsruhe Institute of Technology), Gerhard Neumann (Karlsruhe Institute of Technology)
Meta LearningImageTabular
🎯 What it does: This paper proposes a new Bayesian Meta-Learning (BML) model GMM-NP, which achieves precise inference of task posteriors through Gaussian Mixture Models (GMM) and Trust Region Natural Gradient Variational Inference (TRNG-VI).
Accurate Image Restoration with Attention Retractable Transformer
Jiale Zhang (Shanghai Jiao Tong University), Xin Yuan (Shanghai Jiao Tong University)
RestorationSuper ResolutionCompressionTransformerImage
🎯 What it does: This paper proposes an Attention Retractable Transformer (ART) network for image super-resolution, denoising, and JPEG compression artifact removal, utilizing alternating modules of dense and sparse attention to expand the receptive field and improve recovery quality.
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats
Brian Chmiel (Habana Labs), Daniel Soudry (Habana Labs)
OptimizationConvolutional Neural NetworkTransformerSupervised Fine-TuningImageText
🎯 What it does: A 4-bit full-process training method based on standard formats is designed, utilizing INT4 for weights and activations, and FP4 for gradients, and proposes Logarithmic Unbiased Quantization (LUQ) to achieve unbiased gradient quantization.
Achieve the Minimum Width of Neural Networks for Universal Approximation
Yongqiang Cai (Beijing Normal University)
Ordinary Differential Equation
🎯 What it does: This paper studies the minimum width of neural networks under the Universal Approximation Property (UAP) and provides a general lower bound w*_min = max(d, d_x d_y), proving that this lower bound can be achieved with different activation functions.
Achieving Near-Optimal Individual Regret & Low Communications in Multi-Agent Bandits
Xuchuang Wang (Chinese University of Hong Kong), John C.S. Lui (Chinese University of Hong Kong)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabular
🎯 What it does: A multi-agent collaborative multi-armed bandit algorithm based on UCB and the adjustable communication strategy TCOM (UCB TCOM) is proposed, achieving approximately optimal group/individual regret with only O(log log T) communications;
Achieving Sub-linear Regret in Infinite Horizon Average Reward Constrained MDP with Linear Function Approximation
Arnob Ghosh (Ohio State University), Ness Shroff (Ohio State University)
Reinforcement Learning
🎯 What it does: In the infinite average reward constrained Markov decision process (CMDP), two types of model-free algorithms are proposed, achieving theoretically optimal or approximately optimal lower bounds on sub-linear regret and constraint violation under linear function approximation;
ACMP: Allen-Cahn Message Passing with Attractive and Repulsive Forces for Graph Neural Networks
Yuelin Wang (Shanghai Jiao Tong University), Shi Jin (Shanghai Jiao Tong University)
Graph Neural NetworkGraphOrdinary Differential Equation
🎯 What it does: A message passing mechanism based on Allen-Cahn phase transition dynamics and attractive/repulsive forces (ACMP) is designed and implemented as a trainable neural ODE solver.
Actionable Neural Representations: Grid Cells from Minimal Constraints
Will Dorrell, James C. R. Whittington
Representation Learning
🎯 What it does: Proposes an actionable neural representations framework, combining group representation theory with biological and functional constraints, demonstrating that the optimal representation in two-dimensional space is a multimodal hexagonal grid cell.
Active Image Indexing
Pierre Fernandez (Meta AI), Teddy Furon (Centre Inria de l'Universit' e de Rennes)
RetrievalOptimizationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: By applying small, invisible perturbations to images before publication, the images are brought closer to the quantization center in the feature space, significantly improving the performance of duplicate detection and retrieval under the IVF-PQ index.
Active Learning for Object Detection with Evidential Deep Learning and Hierarchical Uncertainty Aggregation
Younghyun Park (Korea Advanced Institute of Science and Technology), Jaekyun Moon (Korea Advanced Institute of Science and Technology)
Object DetectionImage
🎯 What it does: This paper proposes an active learning framework for object detection that utilizes evidence deep learning to compute the empirical uncertainty of each bounding box and generates information scores for images through a hierarchical uncertainty aggregation method, thereby selecting the most valuable unlabeled samples for annotation.
Active Learning in Bayesian Neural Networks with Balanced Entropy Learning Principle
Jae Oh Woo (Samsung SDS Research America)
ClassificationComputational EfficiencyConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: A new uncertainty measure called Balanced Entropy Acquisition is proposed, applied to active learning in Bayesian neural networks.
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Qingru Zhang (Georgia Institute of Technology), Tuo Zhao (Georgia Institute of Technology)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: An adaptive low-rank fine-tuning method called AdaLoRA is proposed for fine-tuning large pre-trained language models within a limited parameter budget.
Adaptive Optimization in the $\infty$-Width Limit
Etai Littwin (Apple), Greg Yang (Microsoft Research)
Optimization
🎯 What it does: This paper studies the behavior of adaptive optimizers (such as Adam) in the infinite width limit, presents the Adaptive Neural Tangent Kernel (ANTK) and the µ-parameterization in the infinite width limit, and extends the Tensor Programs framework to support adaptive gradient updates.
Adaptive Robust Evidential Optimization For Open Set Detection from Imbalanced Data
Hitesh Sapkota (Rochester Institute of Technology), Qi Yu (Rochester Institute of Technology)
ClassificationOptimizationImage
🎯 What it does: An open set detection method suitable for imbalanced data is proposed—Adaptive Robust Evidential Optimization (AREO), which enhances the learning of minority class samples and the uncertainty discrimination of open set samples by introducing adaptive distributionally robust optimization within the evidence learning framework.
Addressing Parameter Choice Issues in Unsupervised Domain Adaptation by Aggregation
Marius-Constantin Dinu (Johannes Kepler University Linz), Werner Zellinger (Austrian Academy of Sciences)
Domain AdaptationImageTextTime Series
🎯 What it does: In unsupervised domain adaptation, for a given sequence of models obtained from different hyperparameters, this paper proposes a linear aggregation method based on importance-weighted least squares to solve for the optimal linear combination that minimizes target domain error.
Advancing Radiograph Representation Learning with Masked Record Modeling
Hong-Yu Zhou (Xiamen University), Yizhou Yu (University of Hong Kong)
Super ResolutionRepresentation LearningTransformerContrastive LearningImageTextMultimodalityElectronic Health Records
🎯 What it does: This study investigates the use of Masked Record Modeling (MRM) to jointly learn radiological images and their corresponding radiology reports, achieving a more medically informed representation learning.
Adversarial Attacks on Adversarial Bandits
Yuzhe Ma (Microsoft Azure AI), Zhijin Zhou (Amazon)
Adversarial AttackReinforcement LearningTabular
🎯 What it does: This paper studies reward injection attacks in adversarial multi-armed bandits and proposes an attack algorithm that can force any no-regret algorithm to choose the attacker-specified suboptimal arm in almost all rounds without knowing the specific player algorithm, while keeping the attack cost sublinear.
Adversarial Diversity in Hanabi
Brandon Cui (MosaicML), Jakob Nicolaus Foerster
Reinforcement Learning
🎯 What it does: This paper proposes a method based on Adversity (ADVERSITY) to train cooperative strategies that are high-performing, reasonable, and different from known agents in environments like Hanabi and Dec-POMDP.
Adversarial Imitation Learning with Preferences
Aleksandar Taranovic (Karlsruhe Institute of Technology), Gerhard Neumann (Bosch Center for Artificial Intelligence)
Robotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningGenerative Adversarial NetworkTabular
🎯 What it does: A new adversarial imitation learning method (AILP) is proposed, which learns control policies by utilizing both demonstrations and preference feedback.
Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks
Zhiyuan Cheng (Purdue University), Xiangyu Zhang (Purdue University)
Depth EstimationAutonomous DrivingAdversarial AttackContrastive LearningImage
🎯 What it does: This paper proposes an adversarial training method for self-supervised monocular depth estimation models, which can resist physical world attacks without the need for depth ground truth labels. The core idea is to generate realistic stereo image pairs through perspective synthesis, then apply adversarial perturbations with L0-norm constraints on the target image, and utilize reprojection consistency for self-supervised training.
AE-FLOW: Autoencoders with Normalizing Flows for Medical Images Anomaly Detection
Yuzhong Zhao (Shanghai Jiao Tong University), Xiaoqun Zhang (Shanghai Jiao Tong University)
Anomaly DetectionFlow-based ModelAuto EncoderImageBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: A self-supervised AE-FLOW model is proposed, which combines the reconstruction error of the autoencoder with the log-likelihood of the regularized flow for anomaly detection in medical images.
Agent-based Graph Neural Networks
Karolis Martinkus (ETH Zurich), Roger Wattenhofer (ETH Zurich)
ClassificationComputational EfficiencyGraph Neural NetworkAgentic AIGraph
🎯 What it does: This paper proposes AgentNet, a graph-level neural network based on trained neural agents for intelligent roaming on graphs, specifically designed for graph classification tasks.
Agnostic Learning of General ReLU Activation Using Gradient Descent
Pranjal Awasthi (Google Research), Aravindan Vijayaraghavan (Northwestern University)
Optimization
🎯 What it does: For the fitting problem of a single ReLU activation function with a non-zero bias under Gaussian distribution, a convergence analysis of gradient descent in a non-diagonal (agnostic) setting is provided, proving that it can achieve an error of O(OPT) comparable to the optimal ReLU within polynomial steps.
Agree to Disagree: Diversity through Disagreement for Better Transferability
Matteo Pagliardini (École Polytechnique Fédérale de Lausanne), Sai Praneeth Karimireddy (University of California Berkeley)
Domain AdaptationImage
🎯 What it does: A new training framework called D-BAT is proposed, which utilizes the inconsistency on OOD samples to promote the diversity of ensemble models, thereby enhancing the model's transfer performance and uncertainty estimation.
AGRO: Adversarial discovery of error-prone Groups for Robust Optimization
Bhargavi Paranjape (University of Washington), Hannaneh Hajishirzi (University of Washington)
OptimizationTransformerTextBenchmarkAgriculture Related
🎯 What it does: We propose AGRO, an end-to-end method for group distributionally robust optimization that combines a learning model with a soft grouper without the need for pre-defined groups.
AIM: Adapting Image Models for Efficient Video Action Recognition
Taojiannan Yang (University of Central Florida), Mu Li (Amazon Web Services)
RecognitionComputational EfficiencyTransformerVideo
🎯 What it does: The pre-trained image Transformer model is adapted for video action recognition tasks by freezing most parameters and gradually implementing spatial, temporal, and joint spatial-temporal adaptation through the insertion of lightweight Adapter modules.
Aligning Model and Macaque Inferior Temporal Cortex Representations Improves Model-to-Human Behavioral Alignment and Adversarial Robustness
Joel Dapello (Massachusetts Institute of Technology), James J. DiCarlo (Massachusetts Institute of Technology)
Representation LearningAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: Aligning the IT layer representation of the neural network model with the IT neural data of gorillas to make it more similar to brain representations, thereby enhancing the consistency of human behavior and adversarial robustness.
Almost Linear Constant-Factor Sketching for $\ell_1$ and Logistic Regression
Alexander Munteanu (TU Dortmund University), David Woodruff
OptimizationTabular
🎯 What it does: For high-dimensional large-scale data, a new oblivious linear projection (sketch) method is proposed for approximately solving ℓ1 regression and logistic regression (including variance regularization versions), and it provides nearly linear sketch dimensions with constant factor approximation guarantees.
Alternating Differentiation for Optimization Layers
Haixiang Sun (ShanghaiTech University), Dacheng Tao (JD Explore Academy)
OptimizationAuto EncoderTabularTime Series
🎯 What it does: This paper proposes a solver named Alt-Diff, which efficiently performs backpropagation for convex optimization layers with polyhedral constraints in deep neural networks.
Amortised Invariance Learning for Contrastive Self-Supervision
Ruchika Chavhan (University of Edinburgh), Timothy Hospedales (University of Edinburgh)
ClassificationRepresentation LearningConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: A scalable contrastive self-supervised learning method is proposed, which learns adjustable invariance parameters, allowing a single feature extractor to adapt to multiple downstream tasks.
An Adaptive Policy to Employ Sharpness-Aware Minimization
Weisen Jiang (Southern University of Science and Technology), James Kwok (Hong Kong University of Science and Technology)
ClassificationOptimizationComputational EfficiencyConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes an adaptive strategy based on the geometry of the loss surface to decide whether to use Sharpness-Aware Minimization (SAM) or ordinary Empirical Risk Minimization (ERM) during training, thereby constructing two efficient algorithms, AE-SAM and AE-LookSAM, which significantly reduce the computational overhead of SAM and improve model generalization performance.
An Additive Instance-Wise Approach to Multi-class Model Interpretation
Vy Vo (Monash University), Dinh Phung (Monash University)
ClassificationExplainability and InterpretabilityText
🎯 What it does: This paper proposes AIM (Additive Instance-wise Multi-class Interpreter), an end-to-end framework that combines additive explanations and instance-level feature selection to generate local interpretability for multi-class models.
An efficient encoder-decoder architecture with top-down attention for speech separation
Kai Li (Tsinghua University), Xiaolin Hu (Tsinghua University)
RecognitionComputational EfficiencyConvolutional Neural NetworkTransformerAudio
🎯 What it does: An efficient encoder-decoder structure TDANet based on top-down attention is proposed, which achieves excellent performance in speech separation tasks while significantly reducing model complexity.
An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation
Yuqiao Wen (University of Alberta), Lili Mou (University of Alberta)
GenerationTransformerText
🎯 What it does: A method called EqHard-EM, an equal-sized hard EM algorithm, is proposed, which combines multi-decoder and multi-adapter architectures to generate diverse and high-quality dialogue responses.