π― What it does: This paper first identifies and quantifies the ID object contamination issue present in the existing ImageNet-1K OOD test set, and then constructs a completely ID object-free NINCO OOD dataset (5,879 images, 64 classes) and designs various OOD unit tests. Based on this, a systematic evaluation is conducted on multiple models (ViT, ConvNext, EfficientNet, CLIP, etc.) and various OOD detection methods (MSP, MaxLogit, Energy, Mahalanobis, ViM, RMaha, Cosine, etc.).
InGram: Inductive Knowledge Graph Embedding via Relation Graphs
Jaejun Lee (KAIST), Joyce Jiyoung Whang (KAIST)
CodeGraph Neural NetworkGraph
π― What it does: A knowledge graph embedding method named INGRAM is proposed, which can simultaneously generate embedding vectors for new relations and new entities during inference, achieving complete inductive reasoning.
π― What it does: Two new algorithms (MMDP and NRMM) are proposed to accelerate inverse reinforcement learning using expert state distribution, along with theoretical proofs and experimental validation.
Ikechukwu Uchendu (Google), Karol Hausman (Stanford University)
CodeReinforcement LearningTabularBenchmark
π― What it does: This paper proposes Jump-Start Reinforcement Learning (JSRL), which accelerates value-based reinforcement learning by first using an existing guiding policy for roll-in and then allowing the learning policy to continue exploring.
Kernel Sufficient Dimension Reduction and Variable Selection for Compositional Data via Amalgamation
Junyoung Park (Korea Advanced Institute of Science and Technology), Cheolwoo Park (Korea Advanced Institute of Science and Technology)
CodeTabularBiomedical Data
π― What it does: A variable selection method based on amalgamation and sufficient dimension reduction (SDR) is proposed to handle compositional data with a large number of zeros and high dimensionality.
Large Language Models Struggle to Learn Long-Tail Knowledge
Nikhil Kandpal (University of North Carolina), Colin Raffel
CodeTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: This study investigates the relationship between the memory of knowledge in large language models and the frequency of related documents in their pre-training data in closed-book question answering tasks.
π― What it does: Under the framework of self-exploration, a hierarchical state discovery method and the Layered Autonomous Exploration (LAE) algorithm are proposed to achieve efficient learning of an incrementally controllable set of states, significantly reducing sample complexity and applicable to countably infinite state spaces.
LeadFL: Client Self-Defense against Model Poisoning in Federated Learning
Chaoyi Zhu (Delft University of Technology), Lydia Y. Chen (Delft University of Technology)
CodeOptimizationFederated LearningSafty and PrivacyImage
π― What it does: A client-side self-defense method called LeadFL is proposed, which utilizes the regularization of the local gradient Hessian matrix to suppress the persistent effects of model poisoning attacks and can work in conjunction with existing server-side defenses.
π― What it does: This paper studies and proves the learnability of Class Incremental Learning (CIL), and based on this, proposes a new CIL algorithm called ROW, which is based on replay, OOD detection, and WP heads.
π― What it does: This paper studies how to enhance individual fairness and maintain predictive utility by generating comparable samples, referred to as 'antidotes', that resemble the original data distribution on tabular data.
Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks
Dominik Schnaus (Technical University Munich), Rudolph Triebel (German Aerospace Center)
CodeClassificationImage
π― What it does: This paper proposes a learnable prior method based on Bayesian neural networks, using the posterior of trained tasks as the prior for new tasks to enhance generalization and uncertainty estimation.
π― What it does: InstaAug is proposed, a method for automatically generating data augmentation by learning the transformation distribution specific to the input; it can be end-to-end trained together with downstream models during training or learned separately on pre-trained models.
Chinmaya Kausik (University of Michigan), Ambuj Tewari (University of Michigan)
CodeReinforcement LearningTabularTime Series
π― What it does: An offline algorithm is proposed to learn mixed Markov chains/MDPs with only short trajectories, and to perform trajectory clustering and model estimation.
Learning Noisy OR Bayesian Networks with Max-Product Belief Propagation
Antoine Dedieu (DeepMind), Miguel Lazaro-Gredilla
CodeOptimizationReinforcement LearningText
π― What it does: This work proposes a noise-OR Bayesian network (BN) learning framework based on Parallel Max-Product inference, providing a linear time update implementation, and further utilizes GPU acceleration to achieve stochastic gradient training on large-scale dense data.
Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation
Fengxue Zhang (University of Chicago), Yuxin Chen (University of Chicago)
CodeOptimizationAuto EncoderTabular
π― What it does: This paper proposes a Bayesian optimization framework called BALLET based on adaptive hierarchical estimation, which first identifies high-confidence regions of interest (ROI) using a global GP, and then performs fine optimization within that region using a local GP.
Learning the Dynamics of Sparsely Observed Interacting Systems
Linus Bleistein (Inria Paris), Agathe Guilloux (Centre de Recherche des Cordeliers)
CodeTime SeriesOrdinary Differential Equation
π― What it does: This study investigates how to learn the dynamics of a target time series from sparse observations of feature time series, and achieves predictions through controlled differential equations (CDE) and path signature theory.
π― What it does: For semi-supervised learning on multilayer graphs, a parameter-free, data-driven hierarchical aggregation strategy is designed to enhance node classification performance by learning nonlinear weighted combinations across different layers.
π― What it does: A module named Weight Nowcaster Network (WNN) is proposed, which periodically predicts the future weights of the network, allowing for skipping several training epochs and significantly reducing training time.
π― What it does: This paper proposes a model that decomposes complex systems into several relatively independent latent subsystems in sparse and chaotic sequential data, and introduces a meta-system for their interactions.
Learning to Design Analog Circuits to Meet Threshold Specifications
Dmitrii Krylov (University of California), Roy Fox (University of California)
CodeSupervised Fine-TuningTabular
π― What it does: Proposes an automated analog circuit design method based on supervised learning, utilizing simulation data to generate a training set and predict circuit parameters that meet threshold specifications.
π― What it does: A learning to jump framework is proposed, which constructs a deep generative model capable of generating high-quality non-negative sparse data using the dilution and thickening processes of Poisson counting.
π― What it does: A two-stage algorithm is proposed, which learns the unforeseen robustness of the target domain using unseen transformed data from the source domain through an equivariant domain translator, and performs consistency regularization on the translated data to enhance the model's robustness to unknown transformations.
π― What it does: The study utilizes diversified feature representations generated from multiple trainings (with different random seeds) and concatenation (CAT) in multi-distribution and multi-task scenarios, as well as a two-stage fine-tuning process, to enhance the richness and robustness of the representations.
LESSON: Learning to Integrate Exploration Strategies for Reinforcement Learning via an Option Framework
Woojun Kim (Korea Advanced Institute of Science and Technology), Youngchul Sung (Korea Advanced Institute of Science and Technology)
CodeReinforcement Learning
π― What it does: A framework named LESSON is proposed, which utilizes an option model to automatically select and integrate multiple exploration strategies to achieve a balance between exploration and exploitation that changes with the learning phase.
LEVER: Learning to Verify Language-to-Code Generation with Execution
Ansong Ni (Yale University), Xi Victoria Lin (Meta AI)
CodeGenerationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningTextTabular
π― What it does: The LEVER method is proposed, which utilizes a trained verifier to validate and reorder programs generated by large code language models (code-LLM) based on program execution results, thereby improving the quality of language-to-code generation.
LipsNet: A Smooth and Robust Neural Network with Adaptive Lipschitz Constant for High Accuracy Optimal Control
Xujie Song (Tsinghua University), Xiaoming Simon Wang (Didi Chuxing)
CodeOptimizationReinforcement Learning
π― What it does: A neural network structure named LipsNet is proposed, which controls the Lipschitz constant through Multi-dimensional Gradient Normalization (MGN) to achieve action smoothing in RL control strategies.
Shouheng Li (Australian National University), Qing Wang (Australian National University)
CodeClassificationGraph Neural NetworkGraph
π― What it does: This paper proposes a Local Vertex Colouring (LVC) scheme based on graph search and designs a new Search-Guided Graph Neural Network (SGN) based on this scheme. By coloring the search trees generated by BFS/DFS, it extends the expressiveness of 1-WL and can solve problems such as biconnectivity and shortcut graphs that traditional MPNNs cannot distinguish.
MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observations
Anqi Li (University of Washington), Ching-An Cheng (Microsoft Research)
CodeReinforcement Learning
π― What it does: A new paradigm of offline decision learning is proposed - Policy Learning from Observations (PLfO), which unifies offline reinforcement learning and imitation learning from observations.
Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inference
Insung Kong (Seoul National University), Yongdai Kim (Seoul National University)
CodeImageTabularTime Series
π― What it does: A node-sparse Bayesian neural network (mBNN) is proposed, proving that its posterior convergence rate is approximately optimal and adaptively smooth, along with designing a feasible MCMC inference method.
Measuring the Impact of Programming Language Distribution
Gabriel Orlanski (New York University), Michele Catasta (Google)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The BabelCode framework is proposed to achieve multi-language executable evaluation, and a new translation dataset TP3 is built based on it; the Unimax strategy is used to balance the training distribution of 14 languages, training decoder-only models of 1B/2B/4B parameters.
Meta-Learning the Inductive Bias of Simple Neural Circuits
Will Dorrell, Peter E. Latham
CodeMeta LearningSpiking Neural NetworkImageGraph
π― What it does: This paper proposes a meta-learning-based framework that allows an outer meta-learner to assign labels to samples, under which an inner learner is trained. The meta-learner then updates through gradients to minimize the generalization error of the learner on unseen data, thereby extracting the learner's inductive bias (i.e., functions that are easy to generalize).
π― What it does: The Meta-SAGE method enables rapid adaptation of pre-trained reinforcement learning construction models to large-scale combinatorial optimization problems.
CodeTransformerLarge Language ModelPrompt EngineeringMultimodalityBenchmark
π― What it does: Proposes the MEWL benchmark to evaluate the ability of machines to learn word meanings across contexts with reference uncertainty under limited examples.
Ali Taghibakhshi (University of Illinois at Urbana-Champaign), Matthew West (University of Illinois at Urbana-Champaign)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: A multi-level graph neural network (MG-GNN) is proposed to learn a two-layer optimized Restricted Additive Schwarz (ORAS) preconditioner, capable of simultaneously optimizing subdomain Robin boundary conditions and coarse grid interpolation operators.
π― What it does: This study investigates the trajectory curvature of ODE/SDE generative models and proposes a method to reduce curvature by learning the forward process, enabling faster and more accurate sampling.
Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling
Arka Daw (Virginia Tech), Anuj Karpatne (Virginia Tech)
CodeOptimizationPhysics Related
π― What it does: A Retain-Resample-Release (R3) sampling algorithm is proposed to alleviate the propagation failure problem in Physics-Informed Neural Networks (PINN), and its effectiveness is validated.
Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning
Yu Yang (University of California), Baharan Mirzasoleiman (University of California)
CodeObject DetectionSegmentationData-Centric LearningTransformerSupervised Fine-TuningVision Language ModelContrastive LearningImageMultimodality
π― What it does: During the fine-tuning phase of multimodal models (such as CLIP), a language expression-based contrastive loss method is proposed to detect and explicitly remove spurious attributes from the training data, primarily achieved by adding additional spurious contrastive loss at the projection layer.
Yilun Zhu (University of Michigan), Clayton Scott (University of Michigan)
CodeOptimizationReinforcement LearningTabularSequentialBiomedical Data
π― What it does: A more relaxed identifiable condition than the traditional irreducibility assumption is proposed, and based on this condition, a general sampling subsampling (SuMPE) meta-algorithm is designed to estimate the mixing ratio.
Moccasin: Efficient Tensor Rematerialization for Neural Networks
Burak Bartan (Qualcomm Technologies), Bistra Dilkina (University of Southern California)
CodeOptimizationComputational EfficiencyGraph
π― What it does: A Tensor rematerialization method based on constraint programming (MOCCASIN) is proposed, which minimizes the total execution time of the neural network computation graph while satisfying memory budget constraints by retaining interval models.
Model Transferability with Responsive Decision Subjects
Yatong Chen (University of California), Yang Liu (University of California)
CodeDomain AdaptationRecommendation SystemAnomaly DetectionOptimizationTabularFinance Related
π― What it does: The study examines the gap between the performance of a trained model on a new distribution (i.e., induced risk) and the source distribution risk when human decision-makers respond after deploying machine learning models, leading to distribution shift.
Model-based Offline Reinforcement Learning with Count-based Conservatism
Byeongchan Kim (Seoul National University), Min-hwan Oh (Seoul National University)
CodeReinforcement LearningTabularBenchmark
π― What it does: This paper proposes an offline reinforcement learning method called Count-MORL based on a counting-based conservative model, which quantifies model error using the frequency of state-action pairs and implements a conservative policy through reward penalties.
Model-Bellman Inconsistency for Model-based Offline Reinforcement Learning
Yihao Sun (Nanjing University), Yang Yu (Nanjing University)
CodeReinforcement LearningTabularBenchmark
π― What it does: A model-based offline reinforcement learning algorithm called MOBILE is proposed, which utilizes model-Bellman inconsistency to achieve lazy value estimation.
π― What it does: This paper proposes an algorithm called MODeL based on integer linear programming to automatically optimize the lifecycle and memory placement of tensors during neural network training, significantly reducing peak memory usage.
π― What it does: A generative model for denoising diffusion in function space is proposed, aimed at modeling, predicting, and interpolating irregular time series, compatible with continuous time.
Monge, Bregman and Occam: Interpretable Optimal Transport in High-Dimensions with Feature-Sparse Maps
marco cuturi, Pierre Ablin (Apple)
CodeOptimizationExplainability and InterpretabilityDrug DiscoveryBiomedical Data
π― What it does: The study proposes an interpretable high-dimensional optimal transport (OT) mapping method centered on separable sparse regularization, utilizing the combination of Bregman centers and sparse elastic costs to generate adaptive feature-sparse transport vectors.
π― What it does: Proposes the MonoFlow framework, viewing the adversarial training of GANs as vector field learning and sampling of Wasserstein gradient flows.
CodeTransformerLarge Language ModelImageVideoTextMultimodality
π― What it does: Proposes the mPLUG-2 multimodal foundation model, which employs a modular design to achieve unified pre-training for text, images, and videos.
Multi-Fidelity Covariance Estimation in the Log-Euclidean Geometry
Aimee Maurais (Massachusetts Institute of Technology), Youssef Marzouk (Massachusetts Institute of Technology)
Code
π― What it does: A multi-fidelity covariance estimator is proposed, utilizing logarithmic Euclidean geometry to ensure that the estimated matrix is always positive definite;
π― What it does: This paper proposes MH-AIRL, a multi-task hierarchical adversarial inverse reinforcement learning framework based on unlabeled expert demonstrations, capable of learning hierarchical policies suitable for a class of tasks.
π― What it does: A multi-task structure learning framework MTSL is proposed, which utilizes task local similarity to drive the creation and removal of neurons, dynamically learning the multi-task network structure and parameters.
N$\text{A}^\text{2}$Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning
Zichuan Liu (Nanjing University), Chunlin Chen (Nanjing University)
CodeExplainability and InterpretabilityReinforcement Learning
π― What it does: An interpretable multi-agent value decomposition method is proposedβNeural Attention Additive Q-learning (NAQ), which achieves high-order interaction decomposition of joint value and identity semantic-driven credit allocation.
Nearly-Optimal Hierarchical Clustering for Well-Clustered Graphs
Steinar Laenen (University of Edinburgh), He Sun (University of Edinburgh)
CodeGraph Neural NetworkGraph
π― What it does: This paper proposes two nearly linear time hierarchical clustering algorithms that can produce hierarchical trees with costs comparable to the optimal Dasgupta cost for graphs with clear clustering structures, theoretically achieving O(1) approximation while maintaining almost linear time complexity.
NeRFool: Uncovering the Vulnerability of Generalizable Neural Radiance Fields against Adversarial Perturbations
Yonggan Fu (Georgia Institute of Technology), Celine Lin
CodeAdversarial AttackNeural Radiance FieldImage
π― What it does: The NeRFool framework is proposed to systematically analyze the adversarial robustness of GNeRF, providing an attackable NeRFool+ method and designing a defense scheme based on adversarial training.
π― What it does: This paper proposes a continuous-discrete state space model (NCDSSM) capable of handling irregularly sampled time series. It utilizes auxiliary variables to decouple representation learning from dynamics, and achieves precise state estimation and uncertainty quantification through classical continuous-discrete Bayesian inference.
π― What it does: A Markov Jump Process (MJP) model called NeuralMJP is proposed, which is based on neural variational inference and neural ordinary differential equations for posterior inference and parameter estimation of discrete-time observation sequences.
Brian Irwin (University of British Columbia), Avi Ziv (IBM Research)
CodeOptimizationTabularBenchmark
π― What it does: This paper studies a zero-order optimization method called NNAIF that uses neural networks to accelerate implicit filtering, enabling rapid convergence to critical points on noisy black-box functions.
π― What it does: This paper proposes and validates the Distribution Simplicity Bias (DSB): in neural networks trained using stochastic gradient descent, the network first classifies using low-order statistics (mean, covariance) of the input distribution, and then gradually introduces higher-order statistics until optimal classification is achieved.
π― What it does: This paper proposes a Predictive Reasoning Block (PRB) integrated into a residual network called PredRNet, aimed at solving abstract visual reasoning problems such as Raven Progressive Matrices (RPM).
Neural Stochastic Differential Games for Time-series Analysis
Sungwoo Park (LG AI Research), Changhee Lee (Chung-Ang University)
CodeTime SeriesStochastic Differential Equation
π― What it does: This paper proposes a time series prediction framework based on Multi-Agent Stochastic Differential Equations (MaSDE), utilizing cooperative differential games to achieve information aggregation and collaborative decision-making.
π― What it does: A framework for Neural Symbolic Continual Learning (NeSy-CL) is proposed, along with a novel concept-level continual learning strategy called COOL, aimed at maintaining high-quality concepts and avoiding reasoning shortcuts in multi-task settings.
π― What it does: By exchanging the execution order of server-side aggregation and broadcasting, the server and edge devices can work in parallel, eliminating idle time caused by waiting on the server and devices in traditional synchronous federated learning (FL), thus achieving efficient federated learning under heterogeneous devices.
π― What it does: A parameter-free generative model CSWF is proposed, which can directly perform conditional generation and image inpainting through joint Sliced-Wasserstein Flow (SWF).
NP-SemiSeg: When Neural Processes meet Semi-Supervised Semantic Segmentation
Jianfeng Wang (University of Oxford), Thomas Lukasiewicz (Vienna University of Technology)
CodeSegmentationImage
π― What it does: This paper proposes NP-SemiSeg, which introduces the Neural Process framework into semi-supervised semantic segmentation, achieving quantifiable pixel-level uncertainty.
π― What it does: A general framework named NUNO is proposed, which decomposes non-uniform physical data through K-D tree domain decomposition and employs different sizes of uniform grids in each subdomain, followed by using mesh-based neural operators based on FFT or convolution to complete PDE predictions.
Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling
Yuta Saito (Cornell University), Thorsten Joachims (Cornell University)
CodeReinforcement LearningTabular
π― What it does: This study focuses on offline policy evaluation in large action spaces and proposes the OffCEM estimator based on clustering effects and residual effects.
π― What it does: This paper explores the transfer-reward distribution shift problem between offline data and online adaptation in offline meta-reinforcement learning (offline meta-RL), and proposes an 'In-Distribution Online Adaptation with Uncertainty Quantification (IDAQ)' framework based on uncertainty quantification, enabling rapid adaptation to new tasks in the online phase without additional context.
On Data Manifolds Entailed by Structural Causal Models
Ricardo Dominguez-Olmedo (Max Planck Institute for Intelligent Systems), Bernhard SchΓΆlkopf (Max Planck Institute for Intelligent Systems)
CodeOptimizationExplainability and InterpretabilityTabular
π― What it does: This paper studies the data manifold induced by Structural Causal Models (SCM) and constructs a metric based on Riemannian geometry, subsequently using this metric to generate both realistic and causally consistent counterfactual explanations; it also proposes a method for optimization on the manifold to achieve causal algorithmic recourse.
Michal Nauman (University of Warsaw), Marek Cygan (University of Warsaw)
CodeReinforcement LearningSequential
π― What it does: This paper studies how the variance of SPG decreases under multi-action sampling and provides conditions under which using multiple actions is more effective than extending trajectories.
π― What it does: This paper re-evaluates the effectiveness of Learning-from-Scratch (LfS) in visual perception-motor control tasks and makes a fair comparison with frozen pre-trained visual models (PVR, MVP, R3M).
π― What it does: This paper systematically compares three types of sampling methods using approximate transport mappings (normalizing flow), including flow-MCMC, neural-IS, and neutra-MCMC, and conducts empirical and theoretical analyses of their performance under multimodal, unimodal, low-dimensional, high-dimensional, and varying flow learning quality conditions. It also provides a new mixing time upper bound for independent Metropolis-Hastings (IMH) under strongly log-concave targets.
On the Convergence Rate of Gaussianization with Random Rotations
Felix Draxler (Heidelberg University), Ullrich Koethe
CodeFlow-based ModelImage
π― What it does: This paper analyzes the convergence rate of the Gaussianization model under random rotation, proving that the number of layers required in high dimensions is at least linearly related to the dimension.
On the Effectiveness of Offline RL for Dialogue Response Generation
Paloma Sodhi (ASAPP), Ryan McDonald (ASAPP)
CodeTransformerReinforcement LearningText
π― What it does: This paper improves task-oriented dialogue generation models through offline reinforcement learning methods, exploring the effects of teacher forcing, decision transformers, and implicit Q-learning.
On the Identifiability and Estimation of Causal Location-Scale Noise Models
Alexander Immer (ETH Zurich), Alexander Marx (ETH Zurich)
CodeTabular
π― What it does: Proposed and analyzed the causal identifiability of the Location-Scale Noise Model (LSNM), and provided two consistent estimation methods (based on feature mapping and neural networks);
π― What it does: This paper studies the adversarial robustness of randomized ensemble classifiers (REC), providing theoretical limits, necessary and sufficient conditions, and proposes an augmented algorithm named BARRE for training efficient and robust RECs.
On the Within-Group Fairness of Screening Classifiers
Nastaran Okati (Max Planck Institute for Software Systems), Manuel Gomez Rodriguez
CodeTabular
π― What it does: This study examines the unfairness of calibrated screening classifiers within the same group and proposes achieving intra-group monotonicity through post-processing to eliminate this unfairness.
π― What it does: This paper studies the problem of insufficient unimodal features (Modality Laziness) in multimodal learning and proposes two concise post-fusion training strategies based on Unimodal Teacher (UMT) and Unimodal Ensemble (UME).
Pierre Humbert (Universite Paris-Saclay), Sylvain Arlot (Universite Paris-Saclay)
CodeFederated LearningSafty and PrivacyTabular
π― What it does: Proposes a conformal prediction method for constructing effective prediction sets in a one-round communication federated learning environment;
Optimizing Mode Connectivity for Class Incremental Learning
Haitao Wen (University of Electronic Science and Technology of China), Hongliang Li (University of Electronic Science and Technology of China)
CodeOptimizationImage
π― What it does: This paper studies the pattern connectivity between optimal solutions obtained from two consecutive training steps in Class Incremental Learning (CIL) and proposes methods for optimizing the connectivity path (OPC) using Fourier series and gradient projection, as well as EOPC, which is obtained by sampling and averaging in flat regions, serving as a post-processing plugin compatible with various CIL frameworks.
Optimizing NOTEARS Objectives via Topological Swaps
Chang Deng (University of Chicago), Pradeep Kumar Ravikumar
CodeOptimizationGraph
π― What it does: This paper studies a class of non-convex optimization problems and proposes a bi-level optimization algorithm that optimizes the objective function by iteratively swapping pairs of nodes in the topological order of a directed acyclic graph (DAG).
Orthogonality-Enforced Latent Space in Autoencoders: An Approach to Learning Disentangled Representations
Jaehoon Cha (Rutherford Appleton Laboratory), Jeyan Thiyagalingam (Rutherford Appleton Laboratory)
CodeRepresentation LearningAuto EncoderImage
π― What it does: A framework for unsupervised factor separation based on deterministic autoencoders (DAE) is proposed, achieving linear decoupling through Euler encoding and orthogonal latent space.
Oscillation-free Quantization for Low-bit Vision Transformers
Shih-yang Liu, Kwang-Ting Cheng (Hong Kong University of Science and Technology)
CodeClassificationTransformerImage
π― What it does: This paper studies and addresses the issue of weight oscillation that occurs during the quantization of low-bit Vision Transformers. It proposes three techniques: Statistical weight quantization (StatsQ), confidence-guided annealing (CGA), and query-key reparameterization (QKR) to construct a non-oscillating low-bit ViT quantization model.
Out-of-Distribution Generalization of Federated Learning via Implicit Invariant Relationships
Yaming Guo (Jilin University), Yi Chang (Jilin University)
CodeDomain AdaptationFederated LearningImage
π― What it does: This paper proposes the FEDIIR method, which achieves out-of-distribution (OOD) generalization for non-participating clients by implicitly aligning gradients across clients in federated learning to learn invariant relationships.
Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points
Ziye Ma (University of California Berkeley), Somayeh Sojoudi (University of California Berkeley)
CodeRecommendation SystemOptimization
π― What it does: This paper studies the role of over-parameterization in solving non-convex optimization problems, particularly in low-rank matrix recovery problems, proposing a method that forms an infinite hierarchy of non-convex problems through lifting techniques and the Burer-Monteiro decomposition.
Parallel $Q$-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation
Zechu Li (Massachusetts Institute of Technology), Pulkit Agrawal (Massachusetts Institute of Technology)
CodeReinforcement Learning
π― What it does: Proposed the Parallel Q-Learning (PQL) scheme, which utilizes GPU for large-scale parallel simulation on a single workstation to achieve efficient training of offline policy learning;
Jonathan Feldstein (University of Edinburgh), Efthymia Tsamoura (Samsung AI)
CodeRecommendation SystemMixture of ExpertsVideoText
π― What it does: A parallel neural-symbolic integration framework called Concordia is proposed, which combines probabilistic logic theories with deep models, supporting arbitrary probabilistic logics (such as MLN, PSL, etc.).
Parameter-Level Soft-Masking for Continual Learning
Tatsuya Konishi (KDDI Research), Bing Liu (University of Illinois at Chicago)
CodeClassificationOptimizationImage
π― What it does: A soft mask based on parameter gradient importance (SPG) is proposed to achieve continuous learning, which can prevent catastrophic forgetting, encourage knowledge transfer, and reduce network capacity consumption.
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models
Nikolaos Dimitriadis (Ecole Polytechnique Federale de Lausanne), François Fleuret (University of Geneva)
CodeClassificationSegmentationOptimizationImage
π― What it does: This study investigates Pareto frontier learning in multi-task learning and proposes the Pareto Manifold Learning (PaMaL) method, which constructs a continuously adjustable Pareto frontier in a single training session by utilizing linear combinations of single-task models in the weight space.
π― What it does: This paper proposes a patch-level contrastive learning method based on position queries, called PQCL, which achieves patch-level contrast without the need for patch correspondence and enhances the effectiveness of contrastive learning through a cross-attention mechanism.
π― What it does: This paper proposes FedGMMβa framework that utilizes Gaussian Mixture Models (GMM) to model the heterogeneity of joint distributions in federated learning, achieve personalized models, and support uncertainty quantification and new sample detection.
Personalized Federated Learning with Inferred Collaboration Graphs
Rui Ye (Shanghai Jiao Tong University), Yanfeng Wang (Shanghai AI Laboratory)
CodeRecommendation SystemFederated LearningText
π― What it does: A new personalized federated learning framework called pFedGraph is proposed, which utilizes a learnable collaborative graph to determine the degree of collaboration between clients in each communication round, and optimizes the local model on the client side by aggregating the model.
π― What it does: A personalized framework for subgraph federated learning, FED-PUB, is proposed, which can achieve similarity matching and weighted aggregation between subgraphs solely through model parameters without sharing any data, and localizes subgraph-specific weights through sparse masking.
π― What it does: This paper proposes using Prior-Data Fitted Networks (PFNs) as a general surrogate model for Bayesian Optimization (BO), and implements various priors (simple Gaussian processes, HEBO heuristic priors, Bayesian neural network priors) as well as user priors, input scaling, and non-greedy acquisition functions within this framework.
π― What it does: A joint adversarial defense method based on image phase information is proposed, combining phase-level adversarial training and amplitude preprocessing.
PINA: Leveraging Side Information in eXtreme Multi-label Classification via Predicted Instance Neighborhood Aggregation
Eli Chien (University of Illinois), Hsiang-Fu Yu (Amazon)
CodeClassificationTransformerText
π― What it does: The PINA method is proposed, which enhances extreme multi-label classification performance by leveraging label metadata and instance association signals through neighbor prediction aggregation.
Predicting Rare Events by Shrinking Towards Proportional Odds
Gregory Faletto (University of Southern California), Jacob Bien (University of Southern California)
CodeTabularBiomedical Data
π― What it does: A new model called PRESTO is proposed to improve the probability estimation of extremely rare events by utilizing previously more common intermediate category data.