International Conference on Machine Learning Β· 722 papers
Harnessing Heterogeneous Statistical Strength for Personalized Federated Learning via Hierarchical Bayesian Inference
Mahendra Singh Thapa (Rochester Institute of Technology), Rui Li (Rochester Institute of Technology)
CodeFederated LearningTabular
π― What it does: A hierarchical Bayesian inference framework is proposed for simultaneously learning global and personalized models in federated learning.
HashAttention: Semantic Sparsity for Faster Inference
Aditya Desai (University of California Berkeley), Ion Stoica (University of California Berkeley)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelText
π― What it does: By mapping key-value pairs and query vectors to a hash space, sparse attention is achieved by utilizing a learned mapping function and Hamming distance retrieval to identify important tokens in the attention.
π― What it does: A training framework named REST is proposed, which significantly reduces feature staleness and improves the training effectiveness and convergence speed of large-scale graph neural networks by decoupling forward and backward propagation in historical embedding methods and adjusting their execution frequency.
Hi-Patch: Hierarchical Patch GNN for Irregular Multivariate Time Series
Yicheng Luo (South China University of Technology), Qianli Ma (South China University of Technology)
CodeClassificationAnomaly DetectionGraph Neural NetworkTime SeriesBiomedical Data
π― What it does: This paper presents Hi-Patch, a hierarchical patch graph neural network designed for modeling irregular multivariate time series (IMTS) and performing prediction and classification tasks.
π― What it does: This paper presents HIGHT, which utilizes a hierarchical graph tokenizer and hierarchical instruction tuning data to achieve better alignment between molecular graphs and language.
π― What it does: A hierarchical masked autoregressive model Hi-MAR is designed and implemented, using low-resolution image tokens as pivots to first generate global structures and then refine details.
Hierarchical Overlapping Clustering on Graphs: Cost Function, Algorithm and Scalability
Yicheng Pan (Beihang University), Bingchen Fan (Beihang University)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: A new Hierarchical Overlapping Clustering (HOC) cost function is proposed, along with corresponding approximate algorithms and a scalable accelerated version.
Hierarchical Refinement: Optimal Transport to Infinity and Beyond
Peter Halmos (Princeton University), Benjamin Raphael
CodeOptimizationImageTabular
π― What it does: A multi-scale method called Hierarchical Refinement (HiRef) has been developed, which recursively refines dataset partitions using low-rank optimal transport subproblems, ultimately achieving a complete injective Monge mapping while ensuring linear space complexity.
π― What it does: A decoder-only multi-stream model named Hibiki is proposed, capable of real-time synchronous processing of source language speech and generating target language text and speech (S2TT + S2ST).
π― What it does: This paper proposes the Holistic Physics Mixer (HPM), which constructs a new neural operator for solving partial differential equations by simultaneously utilizing spectral structure and point state information in a unified spectral-physical space.
π― What it does: A general framework called MOTIF is proposed to evaluate and enhance the expressiveness of Knowledge Graph Foundation Models (KGFM), clarifying the impact of different motifs on relation invariance learning.
How Much Can Transfer? BRIDGE: Bounded Multi-Domain Graph Foundation Model with Generalization Guarantees
Haonan Yuan (Beihang University), Philip S. Yu (University of Illinois)
CodeClassificationDomain AdaptationGraph Neural NetworkMixture of ExpertsGraph
π― What it does: BRIDGE is proposed, a multi-domain graph-based model that employs domain-invariant aligners, lightweight Mixture of Experts, and spectral regularization for pre-training and prompt fine-tuning to achieve cross-domain graph learning.
HPS: Hard Preference Sampling for Human Preference Alignment
Xiandong Zou (Singapore Management University), Pan Zhou (Singapore Management University)
CodeComputational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: This paper proposes the Hard Preference Sampling (HPS) framework to align the outputs of large language models (LLMs) with human preferences, with a particular focus on the strong rejection of harmful content.
Thore Gerlach (University of Bonn), Nico Piatkowski (Fraunhofer IAIS)
CodeOptimizationBenchmark
π― What it does: Two multi-agent pathfinding (MAPF) algorithms based on quantum-classical hybrid approaches are proposed, namely QUBO-and-Price (QP) and QUBO-and-Cut-and-Price (QCP).
π― What it does: The HYGMA framework is proposed, utilizing dynamic spectral clustering and hypergraph neural networks to achieve adaptive grouping and information transmission for multi-agent coordination.
Ignacio Peis (Technical University of Denmark), Jes Frellsen (Technical University of Denmark)
CodeGenerationData SynthesisTransformerDiffusion modelAuto EncoderImageTime Series
π― What it does: A generative framework LDMI is proposed, which combines implicit neural representations (INR) and latent diffusion models (LDM), and a Transformer-based probabilistic hypernetwork HD is designed for scalable generation and reconstruction of continuous functions.
Hyperbolic-PDE GNN: Spectral Graph Neural Networks in the Perspective of A System of Hyperbolic Partial Differential Equations
Juwei Yue (Institute of Information Engineering, Chinese Academy of Sciences), Li Guo (Institute of Information Engineering, Chinese Academy of Sciences)
CodeExplainability and InterpretabilityRepresentation LearningGraph Neural NetworkGraphStochastic Differential Equation
π― What it does: This paper proposes Hyperbolic-PDE GNN by modeling the message passing of graph neural networks as a set of hyperbolic partial differential equations, allowing node representations to be directly mapped to the solution space spanned by the Laplacian eigenvectors, thereby enhancing interpretability and expressiveness.
Yongxin Yang (Queen Mary University of London), Timothy Hospedales (University of Edinburgh)
CodeGenerationOptimizationComputational EfficiencyTransformerTime SeriesFinance Related
π― What it does: We propose HyperIV, a method for generating arbitrage-free implied volatility surfaces under real-time sparse observations using hypernetworks, with a generation speed of only 2 milliseconds;
IBCircuit: Towards Holistic Circuit Discovery with Information Bottleneck
Tian Bian (Chinese University of Hong Kong), Jia Li (Hong Kong University of Science and Technology)
CodeOptimizationTransformerText
π― What it does: An end-to-end information bottleneck-based circuit discovery framework, IBCircuit, is proposed for the automatic identification of the minimal and most important computational subgraphs in Transformer models.
Identifying biological perturbation targets through causal differential networks
Menghua Wu (Massachusetts Institute of Technology), Tommi Jaakkola (Massachusetts Institute of Technology)
CodeDrug DiscoveryGraph Neural NetworkBiomedical Data
π― What it does: This paper proposes the Causal Differential Networks (CDN) model for identifying intervention targets in gene regulatory networks from observational and interventional data.
Identifying Metric Structures of Deep Latent Variable Models
Stas Syrota (Technical University of Denmark), SΓΈren Hauberg (Technical University of Denmark)
CodeGenerationData SynthesisImage
π― What it does: This paper proposes using Riemannian geometry to identify the metric structure (such as distance, angle, volume) between latent variables in deep latent variable models, rather than the coordinates of individual latent variables, and proves that these metrics can be statistically identified under mild assumptions.
Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent
Santhosh Karnik (Northeastern University), Felix Krahmer (Technical University of Munich)
CodeOptimization
π― What it does: This study investigates the implicit regularization phenomenon of gradient descent in over-parameterized tubal tensor factorization and proves that small random initialization can lead to convergence to low tubal rank solutions.
Improved Off-policy Reinforcement Learning in Biological Sequence Design
Hyeonah Kim (Mila Quebec AI Institute), Jinkyoo Park (KAIST)
CodeDrug DiscoveryConvolutional Neural NetworkTransformerReinforcement LearningBiomedical Data
π― What it does: A Ξ΄-Conservative Search based offline reinforcement learning method is proposed to balance novelty exploration and uncertainty of the surrogate model in biological sequence design.
Improved Theoretically-Grounded Evolutionary Algorithms for Subset Selection with a Linear Cost Constraint
Dan-Xuan Liu (Nanjing University), Chao Qian (Nanjing University)
CodeOptimizationGraph
π― What it does: This paper provides a theoretical reanalysis of the Pareto optimization-based evolutionary algorithm POMC, proving that its approximation ratio for the subset selection problem (monotone submodular functions with linear cost constraints) can be improved to 1/2. It also introduces a novel multi-objective evolutionary algorithm EPOL, which constructs residual problems for each element and utilizes POMC to achieve a practical approximation ratio of 0.6174, demonstrating superior performance compared to existing greedy and evolutionary algorithms in experiments.
Improving LLM Video Understanding with 16 Frames Per Second
Yixuan Li (Tsinghua University), Chao Zhang (Tsinghua University)
CodeRecognitionCompressionOptimizationTransformerLarge Language ModelVision Language ModelVideoMultimodality
π― What it does: F-16 is proposed, a multimodal large language model capable of video understanding at 16 FPS, along with a variable frame rate decoding method;
Improving Out-of-Distribution Detection with Markov Logic Networks
Konstantin Kirchheim (University of Magdeburg), Frank Ortmeier (University of Magdeburg)
CodeAnomaly DetectionExplainability and InterpretabilityImage
π― What it does: A framework for OOD detection that utilizes Markov Logic Networks (MLN) for probabilistic reasoning of interpretable semantic constraints and integrates with existing detectors is proposed.
π― What it does: By selecting pairs of strategies with significant differences in iterative index during the adversarial imitation learning process, preference data that covers a wider range and has higher accuracy is automatically generated, thereby enhancing the generalization ability of the reward model.
Improving the Continuity of Goal-Achievement Ability via Policy Self-Regularization for Goal-Conditioned Reinforcement Learning
Xudong Gong (National University of Defense Technology), Yong Dou (National University of Defense Technology)
CodeRobotic IntelligenceReinforcement Learning
π― What it does: To address the discontinuity issue in the goal achievement capability of Goal-Conditioned Reinforcement Learning (GCRL) algorithms, this paper proposes a threshold-based self-regularization method (MSR) that enhances the continuity of goal achievement by controlling the KL divergence between adjacent goal policies.
In-Context Denoising with One-Layer Transformers: Connections between Attention and Associative Memory Retrieval
Matthew Smart (Flatiron Institute), Anirvan M. Sengupta (Rutgers University)
CodeTransformer
π― What it does: The paper proposes the 'in-context denoising' task, linking the attention mechanism of Transformers with Dense Associative Memory networks, demonstrating that a single-layer Transformer can achieve the Bayes optimal solution for certain denoising problems.
Incorporating Arbitrary Matrix Group Equivariance into KANs
Lexiang Hu (Peking University), Zhouchen Lin (Peking University)
CodePhysics Related
π― What it does: This paper proposes Equivariant Kolmogorov-Arnold Networks (EKAN), which integrates equivariance of arbitrary matrix groups into the KAN architecture to achieve scalable symmetry networks.
Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learning
Anna Soligo (Imperial College London), David Boyle (Imperial College London)
CodeExplainability and InterpretabilityReinforcement LearningTabular
π― What it does: This paper proposes the incorporation of spatial distance-based sparse regularization and neuron migration into the reinforcement learning policy network to facilitate the natural emergence and interpretability of functional modules.
π― What it does: To address the spectral bias problem of multilayer perceptrons in implicit neural representations (INR), a gradient adjustment method induced by the empirical neural tangent kernel (eNTK) is proposed, which adaptively adjusts the NTK spectrum during training to accelerate the convergence of high-frequency components and improve model accuracy.
π― What it does: Proposes the Info-Coevolution framework, which performs sample selection online through information gain estimation and Bayesian prediction fusion, achieving no performance loss with only 68% labeling on ImageNet-1K.
InfoCons: Identifying Interpretable Critical Concepts in Point Clouds via Information Theory
Feifei Li (Fudan University), Min Yang (Fudan University)
CodeAutonomous DrivingExplainability and InterpretabilityPoint Cloud
π― What it does: This paper proposes an explanation framework based on information bottleneck theory, called InfoCons, which can partition point clouds into interpretable key concepts and quantify their causal impact on model predictions.
Information Bottleneck-guided MLPs for Robust Spatial-temporal Forecasting
Min Chen (Tianjin University), Cheng Yan (Tianjin University)
CodeKnowledge DistillationTime SeriesSequential
π― What it does: Proposes the Robust Space-Time Information Bottleneck (RSTIB) principle and implements it as RSTIB-MLP, achieving both robustness and efficiency in space-time prediction tasks with noise disturbances using a pure MLP structure.
π― What it does: Construct an Instance Correlation Graph (ICG) and generate new attributes using VGAE, then weight the new attributes, and finally use Gaussian Naive Bayes for classification on the weighted attributes.
Interpreting CLIP with Hierarchical Sparse Autoencoders
Vladimir Zaigrajew (Warsaw University of Technology), Przemyslaw Biecek (University of Warsaw)
CodeRetrievalExplainability and InterpretabilityAuto EncoderImageTextMultimodality
π― What it does: The Matryoshka SAE architecture is proposed, which achieves hierarchical interpretation and concept extraction of CLIP multimodal representations through multi-layer TopK sparse autoencoders, and is used for similarity search and bias analysis.
π― What it does: Implement integer low-rank adaptation (IntLoRA) on quantized diffusion models, eliminating the post-processing quantization step during inference.
Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning
Changsheng Wang (Michigan State University), Sijia Liu (IBM Research)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: A memory erasure method for LLM based on invariant risk minimization (ILU) is proposed, allowing the model to maintain irrecoverable forgotten information during subsequent fine-tuning.
Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs
Yinong Oliver Wang (Carnegie Mellon University), Nicholas Apostoloff
CodeTransformerLarge Language ModelText
π― What it does: Proposes a fairness metric based on uncertainty, UCerF, and constructs a large-scale gender-occupation co-reference dataset, SynthBias, for fine-grained fairness evaluation of large language models.
ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Tasks
Saurabh Jha (IBM), Ruchir Puri (IBM)
CodeTransformerLarge Language ModelAgentic AITextBenchmarkFinance Related
π― What it does: ITBench has been constructed, an open-source and scalable benchmarking framework for evaluating the performance of AI agents on real IT automation tasks (SRE, CISO, FinOps).
Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations
Lucy Farnik (University of Bristol), Laurence Aitchison (University of Bristol)
CodeTransformerAuto EncoderText
π― What it does: This paper proposes Jacobian Sparse Autoencoders (JSAE) to explicitly sparsify the computation graph of transformer MLP while maintaining activation sparsity.
K$^2$IE: Kernel Method-based Kernel Intensity Estimators for Inhomogeneous Poisson Processes
Hideaki Kim (NTT Corporation), Akinori Fujino (NTT Corporation)
CodeTime Series
π― What it does: This study proposes a kernel intensity estimator (KIE) based on least squares loss for estimating the intensity function of infinite small Poisson processes.
Kernel Quantile Embeddings and Associated Probability Metrics
Masha Naslidnyk (University College London), Krikamol Muandet (CISPA Helmholtz Center for Information Security)
CodeImage
π― What it does: This paper proposes Kernel Quantization Embedding (KQE) and the corresponding Kernel Quantization Divergence (KQD), proving that it can form a probability metric under weaker kernel conditions, and provides a near-linear complexity estimator based on Gaussian measure sampling; it is then compared with methods such as MMD in two-sample testing tasks.
π― What it does: Proposes the KIND pre-training method, which utilizes SVD structure to split model weights into class-independent learn genes and class-specific tailors, allowing the model to be flexibly restructured according to different deployment needs and cross-domain tasks.
KinDEL: DNA-Encoded Library Dataset for Kinase Inhibitors
Benson Chen (Insitro), R. Edward Watts (Insitro)
CodeDrug DiscoveryGraph Neural NetworkTabularBiomedical Data
π― What it does: An 81M molecular DNA-encoded library (DEL) dataset named KinDEL has been released, along with experimental data and offline validation biophysical measurement results for two kinase targets (MAPK14 and DDR1);
π― What it does: A continuous model-based reinforcement learning framework named DRAGO is proposed, which maintains and expands the world model through synthetic experience replay and exploration-based memory recovery.
π― What it does: Proposes the task of 'Knowledge Swapping', which requires selectively forgetting specified knowledge while retaining the core knowledge of the pre-trained model and learning new knowledge.
Kona: An Efficient Privacy-Preservation Framework for KNN Classification by Communication Optimization
Guopeng Lin (Fudan University), Tao Wei (Ant Group)
CodeClassificationSafty and PrivacyComputational EfficiencyTabular
π― What it does: This study proposes the Kona framework, which implements multi-party privacy-preserving KNN classification, addressing the issues of large communication volume and multiple rounds in traditional methods during the online phase.
π― What it does: This paper proposes the KoNODE framework, which models at three hierarchical levels (observation state, ODE parameters, and Koopman linear dynamics) to enable neural ODEs to learn time-evolving parameters and capture the deep linear structure of the system.
π― What it does: A diffusion model based on the Laplace distribution (L-Diffusion) is proposed for pathological image segmentation, achieving differentiation of various tissue/cell types through pixel latent vector contrastive learning.
Label Distribution Propagation-based Label Completion for Crowdsourcing
Tong Wu (China University of Geosciences), Chaoqun Li (China University of Geosciences)
CodeTabular
π― What it does: A label completion algorithm based on label distribution propagation, LDPLC, is proposed to alleviate the sparsity problem of crowdsourced label matrices and improve subsequent label integration effects.
LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
Dachuan Shi (Georgia Tech), Yingyan Celine Lin
CodeGenerationCompressionComputational EfficiencyTransformerLarge Language ModelText
π― What it does: A training-free KV cache optimization framework called LaCache is proposed, which supports long-context continuous generation through a trapezoidal storage pattern and iterative compression.
π― What it does: A label-specific CLIP adapter named LADA is proposed, which extends the CLIP image encoder with lightweight memory vectors to achieve parameter-free continual learning.
π― What it does: This paper presents LAION-C, a novel OOD benchmark specifically designed for large-scale web-crawled datasets (such as LAION), which includes six types of artificially synthesized and highly challenging distortions.
LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Wenzhe Niu (Tianjin University), Chao Hao
CodeTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTime Series
π― What it does: Proposes the LangTime model, which maps multi-domain time series to the LLM semantic space using natural language prompts (Temporal Comprehension Prompts) and fine-tunes through TimePPO reinforcement learning after pre-training to reduce error accumulation in autoregressive predictions.
Language Models May Verbatim Complete Text They Were Not Explicitly Trained On
Ken Liu, Nicolas Papernot (Google)
CodeAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper studies the limitations of n-gram based methods in membership inference of training data for large language models, and demonstrates through experiments that the model can complete text on data that does not contain any n-grams.
LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs β No Silver Bullet for LC or RAG Routing
Kuan Li (Hong Kong University of Science and Technology), Minhao Cheng (Penn State University)
CodeGenerationRetrievalTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
π― What it does: This paper systematically evaluates the performance of Retrieval-Augmented Generation (RAG) and Long Context Large Language Models (LC) on different tasks, model sizes, and context lengths by constructing a new long-text benchmark, LaRA.
Large Language Model-driven Large Neighborhood Search for Large-Scale MILP Problems
Huigen Ye (Tsinghua University), Yaoyang Cheng (Tsinghua University)
CodeOptimizationTransformerLarge Language ModelPrompt EngineeringTabular
π― What it does: This paper proposes a dual-layer self-evolving LNS framework based on large language models (LLM-LNS) for solving large-scale mixed-integer linear programming problems.
CodeTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelTextBenchmarkOrdinary Differential Equation
π― What it does: This paper proposes a fine-tuning method called L2D, which enables pre-trained large language models to scale computational load during inference like diffusion models, thereby improving multi-step reasoning performance.
CodeAnomaly DetectionTransformerLarge Language ModelPrompt EngineeringTextTime Series
π― What it does: A prompt-based framework called LASTS is proposed, which utilizes large language models (LLMs) for multi-tasks such as asynchronous time series prediction, anomaly detection, and missing value imputation.
Latent Imputation before Prediction: A New Computational Paradigm for De Novo Peptide Sequencing
Ye Du (Hong Kong Polytechnic University), Shujun Wang (Hong Kong Polytechnic University)
CodeTransformerBiomedical DataBenchmark
π― What it does: A LIPNovo framework is proposed to first perform potential space missing fragment completion before de novo peptide sequence prediction.
π― What it does: A novel neural operator based on the Latent Mamba Operator, a latent space bidirectional state space model, is proposed for solving high-dimensional PDEs;
π― What it does: We propose LDMol, a latent diffusion model for text-to-molecule generation, which utilizes a structure-aware SMILES encoder and a DiT diffusion network to achieve high-quality text-conditioned molecule generation.
Lean and Mean Adaptive Optimization via Subset-Norm and Subspace-Momentum with Convergence Guarantees
Thien Hang Nguyen (Northeastern University), Huy Nguyen
CodeOptimizationLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes two novel optimizer state compression methods: Subset-Norm (SN) significantly reduces the storage of the second moment matrix by sharing an AdaGrad-Norm style adaptive learning rate across subsets of parameters; Subspace-Momentum (SM) restricts momentum to a low-dimensional subspace while performing SGD directly in the orthogonal complement space, thereby compressing the momentum state.
π― What it does: In the SIS epidemic model with an unknown contact network, a combination of network structure learning and one-time vaccination is proposed to minimize disease extinction time.
π― What it does: This paper proposes Task-Aware Learngene (TAL), a method for predicting parameter initialization based on task characteristics and model scale.
π― What it does: A unified end-to-end training framework LCRON is proposed to optimize the multi-stage collaboration and target mismatch issues in cascade ranking systems.
Yonatan Sommer (Technion - Israel Institute of Technology), Nir Rosenfeld (Technion - Israel Institute of Technology)
CodeClassificationOptimizationTabularFinance Related
π― What it does: The study investigates the scenario of using classifiers to incentivize users to purchase features in the market for positive predictions and proposes a market-aware learning framework.
π― What it does: This paper proposes the use of normalized flows and sliced score matching to learn density-based Fermat distances, and solves geodesics through a relaxation algorithm, addressing the issues of slow convergence and high-dimensional failure in traditional nearest neighbor graph methods.
Learning Distribution-wise Control in Representation Space for Language Models
Chunyuan Deng (Rice University), Hanjie Chen (Rice University)
CodeOptimizationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Proposes to use a distributed (random) intervention method in the representation space of language models instead of traditional point interventions to achieve finer-grained behavior control.
π― What it does: A multi-view fusion state representation framework MFSC is proposed, which utilizes dual affine metric learning and self-attention fusion to generate task-relevant low-dimensional state representations, and enhances robustness to missing or disturbed views through multi-view occlusion and latent reconstruction auxiliary tasks.
Learning Initial Basis Selection for Linear Programming via Duality-Inspired Tripartite Graph Representation and Comprehensive Supervision
Anqi Lu (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: A GNN model based on a tripartite graph is proposed for learning the initial basis selection of linear programming, and the prediction quality is enhanced through comprehensive supervision.
Learning Input Encodings for Kernel-Optimal Implicit Neural Representations
Zhemin Li (National University of Defense Technology), Xiaolong Han (National University of Defense Technology)
CodeRetrievalNeural Radiance FieldImage
π― What it does: A kernel alignment-based input encoding learning framework (PEAK) is proposed, which enhances the generalization ability of implicit neural representations (INR) by aligning the neural tangent kernel of infinitely wide networks with the theoretically optimal kernel.
Learning Invariant Causal Mechanism from Vision-Language Models
Zeen Song (Institute of Software Chinese Academy of Sciences), Wenwen Qiang (Institute of Software Chinese Academy of Sciences)
CodeDomain AdaptationTransformerVision Language ModelContrastive LearningImageTextBenchmark
π― What it does: This paper proposes learning invariant causal mechanisms on the Vision-Language pre-trained model CLIP to enhance OOD generalization.
Learning Latent Graph Structures and their Uncertainty
Alessandro Manenti (Universita della Svizzera italiana), Cesare Alippi (Politecnico di Milano)
CodeGraph Neural NetworkGraph
π― What it does: For prediction tasks with unknown graph structures, a method is proposed that combines learning prediction models and latent graph distributions, utilizing distribution matching loss to achieve calibration of the latent graph and optimal point prediction.
Learning Monotonic Probabilities with a Generative Cost Model
Yongxiang Tang (Kuaishou), Peng Jiang
CodeOptimizationAuto EncoderTabular
π― What it does: This paper proposes a generative framework that transforms monotonic probability modeling into latent cost variable modeling, constructing two models: the strictly monotonic Generative Cost Model (GCM) and the non-strictly monotonic Implicit GCM (IGCM);
π― What it does: A multi-agent curriculum learning method based on learning progress is proposed, which automatically adjusts the number of agents to overcome the sparse reward problem.
Learning Single Index Models with Diffusion Priors
Anqi Tang (University of Electronic Science and Technology of China), Zhaoqiang Liu (University of Electronic Science and Technology of China)
CodeRestorationDiffusion modelImage
π― What it does: Using diffusion models as a prior, efficient signal recovery of the single-index model (SIM) is achieved through reverse sampling starting from intermediate time points.
π― What it does: A parameterized survival analysis model based on the Asymmetric Laplace Distribution (ALD) is proposed, which directly learns the distribution parameters for each individual through maximum likelihood, allowing for closed-form calculations of statistics such as mean, median, variance, and quantiles.
π― What it does: This paper proposes a meta-learning-based two-layer optimization frameworkβMeta-Quantization, which re-parameterizes the codebook using a hyper-net and learns it within vector quantization networks to achieve more efficient and task-related quantization.
Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL
Qin-Wen Luo (Nanjing University of Aeronautics and Astronautics), Sheng-Jun Huang (Nanjing University of Aeronautics and Astronautics)
CodeReinforcement Learning
π― What it does: A selective state adaptive regularization method is proposed to enhance the reliability of Bellman updates in offline reinforcement learning.
Lego Sketch: A Scalable Memory-augmented Neural Network for Sketching Data Streams
Yuan Feng (University of Science and Technology of China), S Kevin Zhou
CodeComputational EfficiencyMeta LearningTime Series
π― What it does: This paper proposes Lego Sketch, a scalable memory-augmented neural network architecture for frequency estimation of infinite data streams within limited space.
π― What it does: A label error detection method based on multimodal neighbors, LEMON, is proposed, which utilizes a multimodal model pre-trained with contrastive learning (such as CLIP) to embed image-text pairs, combining multimodal similarity and the nearest neighbor distances of images/texts to calculate mislabeling scores.
π― What it does: This work proposes a complete offline-online end-to-end method that utilizes offline data to estimate the low-dimensional subspace of linear potential contextual bandits (SOLD), and based on this, designs the optimal online algorithm LOCAL-UCB and the feasible algorithm ProBALL-UCB, completing the proof of theoretical upper bounds and matching lower bounds, and providing a proof of the de Finetti theorem to demonstrate the generality of potential bandits.
CodeOptimizationExplainability and InterpretabilityReinforcement LearningTabular
π― What it does: Transform decision trees into Boolean logic expressions (DNF) to eliminate predictive equivalence and apply it in multiple machine learning tasks.
LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning
Zihang Liu (University of California), Shiwei Liu (University of Oxford)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A sparse fine-tuning method called LIFT is proposed, which updates only the maximum modulus parameters obtained after low-rank approximation, thereby achieving efficient inference fine-tuning on large models.
π― What it does: A drag-and-drop image editing method that utilizes a video-trained conditional generative model to achieve approximately 1 second of editing time is proposed.
Lightweight Protocols for Distributed Private Quantile Estimation
Anders Aamand (University of Copenhagen), Rasmus Pagh (University of Copenhagen)
CodeSafty and Privacy
π― What it does: Two adaptive algorithms are proposed for estimating the quantiles (especially the median) of distributed datasets under the Local Differential Privacy (LDP) and Shuffle Differential Privacy (shuffleDP) models, with sample complexities of O(log B/(Ρ²α²)) and O((1/Ξ±Β²+1/Ρ²)Β·log B), respectively.
π― What it does: This study investigates the linear mode connectivity (LMC) of neural networks under permutation symmetry and proposes a straight-through estimator for multi-models (STE-MM) to find parameter arrangements that can aggregate multiple independently trained models into the same low-loss convex basin; experiments are then conducted on various model architectures.
LineFlow: A Framework to Learn Active Control of Production Lines
Kai MΓΌller, Tobias Windisch (University of Applied Sciences Kempten)
CodeOptimizationReinforcement Learning
π― What it does: This paper presents the open-source LineFlow framework for high-precision simulation of production lines and training reinforcement learning (RL) agents for active control.
LLaVA-ReID: Selective Multi-image Questioner for Interactive Person Re-Identification
Yiding Lu (Sichuan University), Xi Peng (Sichuan University)
CodeRecognitionRetrievalTransformerLarge Language ModelImageTextMultimodalityRetrieval-Augmented Generation
π― What it does: An interactive person re-identification (Inter-ReID) framework is proposed, which continuously refines descriptions through multi-turn dialogue with witnesses to achieve more accurate person retrieval.
LLM-Assisted Semantically Diverse Teammate Generation for Efficient Multi-agent Coordination
Lihe Li (Nanjing University), Yang Yu (Nanjing University)
CodeRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningAgentic AIText
π― What it does: Utilizing large language models (LLM) to generate descriptive and diverse collaborative behaviors, automating the construction of corresponding reward functions, thereby training semantically diverse teammate strategies, and using a multi-head network to continuously train a main control agent capable of matching different teammates.
LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models
Parshin Shojaee (Virginia Tech), Chandan K. Reddy
CodeTransformerLarge Language ModelTextBenchmarkPhysics Related
π― What it does: A benchmark called LLM-SRBench has been established to evaluate the performance of large language models in the task of scientific equation discovery.
LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models
Marwa Abdulhai (University of California), Sergey Levine (University of California)
CodeTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
π― What it does: This paper proposes the LMRL-Gym evaluation benchmark, which includes 8 multi-turn RL tasks (3 interactive dialogues and 5 text games), and provides an open-source framework for researchers to quickly conduct multi-turn reinforcement learning experiments with LLMs.