International Conference on Machine Learning Β· 550 papers
LLM Maybe LongLM: SelfExtend LLM Context Window Without Tuning
Hongye Jin (Texas A&M University), Xia Hu (Rice University)
CodeTransformerLarge Language ModelText
π― What it does: The SelfExtend method is proposed, which expands the context window of large language models without fine-tuning during the inference phase by mapping unseen relative positional information and combining hierarchical attention (grouped attention and adjacent normal attention).
LLM-Empowered State Representation for Reinforcement Learning
Boyuan Wang (Tsinghua University), Xiangyang Ji (Tsinghua University)
CodeLarge Language ModelReinforcement LearningTabularBenchmark
π― What it does: Utilizing LLM to automatically generate task-related state representations and intrinsic reward functions, improving the quality of state representation in RL and accelerating learning.
Local Causal Structure Learning in the Presence of Latent Variables
Feng Xie (Beijing Technology and Business University), Zhi Geng (Beijing Technology and Business University)
CodeGraphTabular
π― What it does: An algorithm is proposed to accurately discover the direct causal relationships of a given target variable using local structure learning (MMB-by-MMB) in the presence of latent variables.
π― What it does: This paper proposes a new upper bound for the expected softplus function and applies it to variational logistic regression and Gaussian process classification, introducing the VI-PER method.
π― What it does: This paper demonstrates that full fine-tuning leads to a decline in tail class performance in long-tail learning tasks by comparing full fine-tuning and lightweight fine-tuning, and proposes a lightweight fine-tuning framework called LIFT, which can achieve high accuracy with very few learnable parameters in a short time.
Longitudinal Targeted Minimum Loss-based Estimation with Temporal-Difference Heterogeneous Transformer
Toru Shirakawa (Osaka University), Mark J. van der Laan
CodeTransformerTime SeriesSequential
π― What it does: This paper proposes a deep longitudinal TMLE method based on Transformer (Deep LTMLE) for estimating causal average outcomes under dynamic interventions.
π― What it does: The theoretical analysis of LoRA (Low-Rank Adaptation) fine-tuning under the NTK (Neural Tangent Kernel) framework proves the existence of a global optimal solution with a rank of βN, and when the rank r β³ βN, the optimization landscape has no spurious local minima, allowing gradient descent to converge to a low-rank global optimum, which also exhibits good generalization performance.
LQER: Low-Rank Quantization Error Reconstruction for LLMs
Cheng Zhang (Imperial College London), Yiren Zhao (Imperial College London)
CodeCompressionOptimizationComputational EfficiencyTransformerLarge Language ModelText
π― What it does: A new post-training quantization framework LQER is proposed, which combines quantization and low-rank approximation to reconstruct the quantization error of weights, thereby restoring the performance of large language models (LLMs) without additional training or iterative optimization.
LSEnet: Lorentz Structural Entropy Neural Network for Deep Graph Clustering
Li Sun (North China Electric Power University), Philip S. Yu (University of Illinois at Chicago)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: A graph clustering method called LSEnet is proposed, which does not require a predefined number of clusters. It utilizes differentiable structural information (DSI) to learn a hierarchical partition tree in the hyperbolic space of the Lorentz model and complete clustering.
π― What it does: The MAESTRO framework is proposed, which dynamically learns the optimal rank of each layer during training through Low-Rank Ordered Decomposition (LOD), achieving low-rank compression of the model.
MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models
Justin Chen, Mohit Bansal (University of North Carolina Chapel Hill)
CodeKnowledge DistillationGraph Neural NetworkLarge Language ModelContrastive LearningTextGraph
π― What it does: Construct a Multi-Agent Generated Graph (MAG) and distill its structure into a small language model to enhance reasoning capabilities.
MAGNOLIA: Matching Algorithms via GNNs for Online Value-to-go Approximation
Alexandre Hayderi (Stanford University), Anders Wikum (Stanford University)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: This paper proposes an online Bayesian bipartite matching algorithm called MAGNOLIA based on graph neural networks (GNN), which learns to estimate value-to-go (VTG) using GNN to make approximately optimal matching decisions upon the arrival of each online node.
π― What it does: The PRESTO framework is proposed to map and compare the latent space structures of different machine learning models (especially those using latent representations) in a multiverse.
Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs
Ling Yang (Peking University), Bin CUI
CodeGenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageText
π― What it does: A training-free framework called RPG (Recaption, Plan, Generate) is proposed, utilizing multimodal large language models (MLLM) for text re-labeling, planning regions, and achieving region-level generation and editing within diffusion models.
π― What it does: In a simultaneous cooperation and competition game, the Albatross method is proposed, which can adaptively make decisions based on the opponent's bounded rationality under zero-shot interaction.
MD tree: a model-diagnostic tree grown on loss landscape
Yefan Zhou (Dartmouth College), Yaoqing Yang (Zhejiang University)
CodeOptimizationExplainability and InterpretabilityConvolutional Neural NetworkImage
π― What it does: This paper proposes a model diagnostic tree (MD tree) based on loss landscape metrics, which can diagnose the sources of failure in pre-trained neural networks (such as optimizer hyperparameters, model size, or insufficient data) without retraining and without accessing the original training configuration.
Mean-field Underdamped Langevin Dynamics and its Spacetime Discretization
Qiang Fu (Yale University), Ashia Camage Wilson
CodeOptimizationStochastic Differential Equation
π― What it does: A N-particle algorithm (N-ULA) based on average field undamped Langevin dynamics is proposed for high-dimensional entropy-regularized mean field optimization problems.
π― What it does: This paper focuses on constructing space-filling designs for discrete (categorical) data spaces and proposes a subset selection method based on combinatorial optimization.
π― What it does: This paper proposes Mechanistic Neural Networks (MNN), which can explicitly learn and solve differential equations by embedding Mechanistic Blocks in neural networks, thereby discovering and simulating dynamic systems directly from data.
CodeGenerationData SynthesisComputational EfficiencyMeta LearningImageTime Series
π― What it does: This study proposes a Constant Memory Attention Block (CMAB) and integrates it into Neural Processes (NP), forming Constant Memory Attentive Neural Processes (CMANP), achieving an NP model that uses constant memory and constant computation during the conditioning, querying, and updating phases.
π― What it does: A Meta-RL method named PSBL is proposed, which achieves robustness to task distribution drift by utilizing a pre-trained transformer (LILTrans) for gradient-free online inference during the testing phase.
CodeClassificationAnomaly DetectionRepresentation LearningRecurrent Neural NetworkContrastive LearningTime SeriesFinance Related
π― What it does: Designed and implemented MF-CLR, a self-supervised contrastive learning framework for learning universal representations of multi-frequency time series.
π― What it does: A model versioning and management system named MGit has been designed and implemented, which records model evolution traces and provides storage optimization, testing, updating, and collaboration features.
π― What it does: A new observation-based imitation learning algorithm called MAAD is proposed, which infers missing expert actions using an inverse dynamics model and guides policy learning through behavior cloning regularization.
Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss
Zhenlong Liu (Southern University of Science and Technology), Hongxin Wei (Southern University of Science and Technology)
CodeOptimizationSafty and PrivacyConvolutional Neural NetworkImage
π― What it does: The Convex-Concave Loss (CCL) is proposed by incorporating a concave function into the cross-entropy loss, significantly improving the variance of training sample loss, thereby reducing the advantage of membership inference attacks (MIA) while maintaining model performance.
π― What it does: A modular adversarial training algorithm, Modular-DCM, is proposed for constructing deep causal generative models (DCM), enabling accurate estimation of identifiable causal effects and causal sampling in high-dimensional data (such as images) with potential confounding factors.
π― What it does: This paper proposes a two-layer framework MOKD based on optimized kernel dependency for cross-domain few-shot classification fine-tuning.
π― What it does: This paper proposes a 3D molecular representation learning framework called MOL-AE based on autoencoders, and designs a new 3D Cloze Test training objective to address the inconsistency between pre-training and downstream tasks in traditional coordinate denoising frameworks, as well as the twisted optimization problem caused by coordinate denoising.
MolCRAFT: Structure-Based Drug Design in Continuous Parameter Space
Yanru Qu (Tsinghua University), Wei-Ying Ma (Tsinghua University)
CodeDrug DiscoveryGraph Neural NetworkFlow-based ModelBiomedical Data
π― What it does: A generative model called MolCRAFT is proposed for structure-based drug design in continuous parameter space, addressing the issues of mode collapse and discrete-continuous space mismatch in traditional autoregressive and diffusion models for 3D conformation generation.
Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
Long Qian (Zhejiang University), Siliang Tang (National University of Singapore)
CodeRecognitionRetrievalTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextMultimodality
π― What it does: Momentor has been developed, a video large language model with fine-grained temporal reasoning capabilities, trained on the automatically generated Moment-10M dataset;
π― What it does: A Momentum Particle Descent (MPD) algorithm is designed, introducing a momentum mechanism into particle gradient descent for maximum likelihood estimation of latent variable models.
Monotone, Bi-Lipschitz, and Polyak-Εojasiewicz Networks
Ruigang Wang (University of Sydney), Ian Manchester
CodeOptimizationImage
π― What it does: This paper proposes BiLipNet (reversible bi-Lipschitz network) and PLNet (scalar output network satisfying the Polyak-Εojasiewicz condition), and presents a method to construct reversible residual layers while ensuring strong monotonicity and Lipschitz continuity.
π― What it does: This paper studies the overfitting problem of replay samples in continual learning and proposes a Multi-layer Replay Feature Augmentation (MRFA) method, which enhances the features of replay samples through gradient ascent at each layer to increase the inter-layer margin, thereby alleviating catastrophic forgetting.
Multi-Patch Prediction: Adapting Language Models for Time Series Representation Learning
Yuxuan Bian (Chinese University of Hong Kong), Qiang Xu (Chinese University of Hong Kong)
CodeAnomaly DetectionRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTime Series
π― What it does: Transforming time series forecasting into a self-supervised multi-patch prediction task, utilizing causal Transformer (GPT2) to learn time series representations under a two-stage pre-training + fine-tuning framework.
π― What it does: This paper presents MusicFlow, a cascaded text-to-music generation model based on flow matching, capable of generating music from natural language descriptions and supporting tasks such as continuation and filling in the blanks.
MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts
Jianan Zhou (Nanyang Technological University), Xu Chi
CodeOptimizationTransformerReinforcement LearningMixture of Experts
π― What it does: A multi-task vehicle routing problem solver MVMoE has been developed, utilizing a mixture of experts (MoE) to achieve unified learning and solving for various VRP variants.
π― What it does: The GEOM method is proposed to achieve lossless graph condensation, first using curriculum learning to generate diverse supervisory signals, and then effectively transferring this information to small synthetic graphs through extended window matching, supplemented by a knowledge embedding extractor to further enhance performance.
π― What it does: A score estimator based on nearest neighbors is proposed, which uses multiple training samples for self-normalized importance sampling to reduce the variance of score estimation and decrease bias.
Neighboring Perturbations of Knowledge Editing on Large Language Models
Jun-Yu Ma (University of Science and Technology of China), Jia-Chen Gu (University of California)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: This paper studies the perturbation of neighboring knowledge when adding new knowledge to large language models and proposes an additive metric for measuring such perturbations, along with the evaluation benchmark PEAK, and a pluggable editing framework APP, which aims to reduce the forgetting of existing answer lists and the introduction of noise while adding knowledge.
Nesting Particle Filters for Experimental Design in Dynamical Systems
Sahel Iqbal (Aalto University), Hany Abdulsamad (Aalto University)
CodeOptimizationReinforcement LearningTime Series
π― What it does: A nested particle filtering method (InsideβOut SMC 2) is proposed for Bayesian experimental design with non-exchangeable data, and gradient optimization is performed within the particle MCMC framework.
Shuai Zhang (Academy of Mathematics and Systems Science, Chinese Academy of Sciences), Zhi-Ming Ma (Academy of Mathematics and Systems Science, Chinese Academy of Sciences)
π― What it does: A new framework is proposed that views intensity processes as solutions to Neural Jump Diffusion Stochastic Differential Equations (NJDSDE), addressing the limitation of traditional Temporal Point Processes (TPP) that require a predefined intensity form.
Neural SPH: Improved Neural Modeling of Lagrangian Fluid Dynamics
Artur Toshev, Johannes Brandstetter (Johannes Kepler University)
CodeGraph Neural NetworkGraphPhysics Related
π― What it does: This paper proposes the Neural SPH framework, which enhances the physical consistency and long-term stability of the GNN simulator in Lagrangian fluid dynamics by incorporating external force handling and SPH relaxation steps during the training and inference phases.
NeWRF: A Deep Learning Framework for Wireless Radiation Field Reconstruction and Channel Prediction
Haofan Lu (University of California Los Angeles), Omid Abari (University of California Los Angeles)
CodeNeural Radiance FieldPoint Cloud
π― What it does: Proposes the NeWRF framework, which learns indoor wireless channel fields using sparse measurement points and accurately predicts channels at unmeasured points.
No Double Descent in Principal Component Regression: A High-Dimensional Analysis
Daniel Gedon (Uppsala University), Thomas B. SchΓΆn (Uppsala University)
CodeTabularAgriculture Related
π― What it does: The theoretical analysis of the generalization risk of principal component regression (PCR) in high-dimensional data under the spiked covariance model is conducted, proving that there is no double descent phenomenon.
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates
Riccardo Grazzi (Istituto Italiano di Tecnologia), Saverio Salzo (UniversitΓ La Sapienza)
CodeOptimizationHyperparameter SearchTabular
π― What it does: This paper studies the fixed point differentiation problem for non-differentiable contraction mappings, providing deterministic convergence rates for ITD (Iterative Differentiation) and AID (Approximate Implicit Differentiation), and proposes the first provably convergent stochastic method, NSID; experiments are then conducted to validate these results in bilevel optimization (hyperparameter optimization and data poisoning).
Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness
Samir Khan (Stanford University), Johan Ugander (Stanford University)
CodeTabular
π― What it does: Under the condition of no overlap, a precise partial identification framework for offline policy evaluation (OPE) is proposed using the smoothness (Lipschitz) assumption, along with a closed-form linear programming solution method; asymptotically optimal estimators and convergence rate analysis are also provided.
CodeRobotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningGenerative Adversarial NetworkImage
π― What it does: A framework for imitation learning from offline pre-training to online fine-tuning, called OLLIE, is proposed. It learns an approximate expert policy from offline data and synchronously obtains an aligned discriminator, supporting rapid online fine-tuning for high-dimensional tasks.
On Hypothesis Transfer Learning of Functional Linear Models
Haotian Lin (Pennsylvania State University), Matthew Reimherr (Pennsylvania State University)
CodeTabularTime SeriesFinance Related
π― What it does: This paper proposes and theoretically analyzes two algorithms, TL-FLR and ATL-FLR, based on Optimal Transfer Learning (OTL) within the frameworks of Functional Linear Regression (FLR) and Functional Generalized Linear Models (FGLM), aimed at enhancing the predictive performance of the target task when information from the source task is limited.
π― What it does: Two methods, LOCOGEN and LOCOEDIT, are proposed for locating and editing the control layers of visual attributes in text-to-image generation models.
On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization
Motahareh Sohrabi (Mila-Quebec AI Institute and Universite de Montreal), Jose Gallego-Posada (Mila-Quebec AI Institute and Universite de Montreal)
CodeOptimizationTabular
π― What it does: A Ξ½PI algorithm based on a PI controller is proposed for updating Lagrange multipliers in Lagrangian constraint optimization, along with theoretical analysis and experimental validation.
On Prompt-Driven Safeguarding for Large Language Models
Chujie Zheng (Tsinghua University), Nanyun Peng (University of California Los Angeles)
CodeOptimizationSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Study the working mechanism of safety prompts in large language models and propose a representation space-based safety prompt optimization method called DRO.
On the Convergence of Projected Bures-Wasserstein Gradient Descent under Euclidean Strong Convexity
Junyi FAN, Zhengyuan Zhou
CodeOptimization
π― What it does: The paper studies the global convergence theory of the Bures-Wasserstein gradient descent algorithm (including the projection version) for positive definite matrices under Euclidean strong convexity and smoothness conditions, and provides a closed-form projection formula for the BW ball constraint.
On the Diminishing Returns of Width for Continual Learning
Etash Kumar Guha, Vihan Lakshman (ThirdAI)
CodeConvolutional Neural NetworkTransformerImage
π― What it does: This study investigates the impact of width on catastrophic forgetting in continual learning, proposes a theoretical framework, and demonstrates that increasing width leads to diminishing returns, validating its performance on large-width FFN and WideResNet.
π― What it does: This paper proposes and unifies the spectral invariant graph neural network framework, namely Eigenspace Projection GNN (EPNN), and conducts a theoretical analysis of its expressive power;
On the Feasibility of Single-Pass Full-Capacity Learning in Linear Threshold Neurons with Binary Input Vectors
Ruipeng Liu (Syracuse University), Garrett Ethan Katz (Syracuse University)
CodeSpiking Neural NetworkTabular
π― What it does: This study investigates the feasibility of single-pass and full-capacity linear threshold neuron learning rules, focusing on binary input vectors.
On the Hardness of Probabilistic Neurosymbolic Learning
Jaron Maene (KU Leuven), Luc De Raedt (Orebro University)
CodeTabular
π― What it does: This paper studies the complexity and feasibility of approximating the gradient of Weighted Model Counting (WMC) in probabilistic neural-symbolic learning.
π― What it does: This study investigates the minimum order bias of random feature models (RF) and Transformers in the context of non-Boolean functions' GOTU (generalization to unseen domains), exploring two scenarios: small features and sparse targets. It proves that under the small feature paradigm, RF converges to the lowest order interpolator and discusses the peculiarities of square root unit embeddings.
π― What it does: This paper studies the geometric laws of ODE-based diffusion model sampling trajectories and proposes a time scheduling method based on these laws.
π― What it does: An adaptive prompt tuning framework called AdaPromptCL is proposed to handle varying degrees of semantic drift in continual learning.
Online Learning and Information Exponents: The Importance of Batch size & Time/Complexity Tradeoffs
Luca Arnaboldi (Ecole Polytechnique Federale de Lausanne), Ludovic Stephan (Ecole Polytechnique Federale de Lausanne)
CodeOrdinary Differential Equation
π― What it does: This paper studies the impact of batch size on the iteration time and sample complexity when training a two-layer neural network with one-shot SGD, and proposes a method called 'correlated loss SGD' to overcome the time bottleneck caused by the self-interaction of traditional SGD in the case of large batches.
Open-Domain Text Evaluation via Contrastive Distribution Methods
Sidi Lu (University of California), Nanyun Peng (University of California)
CodeGenerationData SynthesisTransformerLarge Language ModelContrastive LearningText
π― What it does: A contrastive evaluation framework based on two language models of different scales is proposedβContrastive Distribution Methods (CDM), which includes generative and discriminative variants, to generate pseudo-negative samples or directly calculate contrastive momentum, thereby obtaining reference-free text quality scores.
π― What it does: A Distance-Aware Calibration (DAC) method is proposed to correct the confidence distortion of fine-tuned CLIP under open vocabulary.
Operator SVD with Neural Networks via Nested Low-Rank Approximation
Jongha Jon Ryu (Massachusetts Institute of Technology), Gregory W. Wornell (Massachusetts Institute of Technology)
CodeRetrievalOptimizationImagePhysics Related
π― What it does: A framework utilizing Nested Low-Rank Approximation (Nested LoRA) is proposed, which uses neural networks to directly learn the first L singular values and corresponding singular functions of linear operators (such as integral kernels or differential operators);
Xuanfei Ren (University of Science and Technology of China), Pan Xu (Duke University)
CodeOptimizationReinforcement LearningTabular
π― What it does: This paper proposes a new batch linear Bandit algorithm E4, which adopts the Explore-Estimate-Eliminate-Exploit framework to address the optimal decision-making problem under limited time and asymptotic limits.
Optimal Eye Surgeon: Finding image priors through sparse generators at initialization
Avrajit Ghosh (Michigan State University), Rongrong Wang (Michigan State University)
CodeRestorationImageMagnetic Resonance Imaging
π― What it does: This paper proposes a framework for sparse pruning during the random initialization of networks, resulting in a sparse subnetwork (Sparse-DIP) that can serve as an image prior for image recovery with little or no training.
Optimally Improving Cooperative Learning in a Social Setting
Shahrzad Haddadan (Rutgers University), Jie Gao (Rutgers University)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: The study investigates how to select k agents with their own classifiers in a social network to improve their original predictions, thereby maximizing the overall prediction accuracy of the network.
π― What it does: A framework is proposed that incorporates optimistic updates into multi-agent policy gradient methods, primarily achieved by clipping negative values in the advantage function to 0 (or using Leaky ReLU);
OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models
Ali AhmadiTeshnizi (Stanford University), Madeleine Udell (Stanford University)
CodeOptimizationTransformerLarge Language ModelAgentic AIPrompt EngineeringText
π― What it does: A multi-agent system named OptiMUS based on large language models has been developed, capable of automatically decomposing optimization problems described in natural language into parameters, constraints, and objectives, generating executable solving code, and self-debugging, ultimately solving linear programming and mixed-integer programming problems.
π― What it does: This paper proposes an OSN framework that learns all feasible dynamic 3D scene representations from monocular RGB videos, capable of infinitely generating 3D scene configurations that satisfy the video.
π― What it does: The contrastive learning and inference processes of CLIP are treated as inverse optimal transport (Inverse OT) bi-level optimization and graph matching problems, proposing the OT-CLIP loss family and OT prediction framework.
Out of the Ordinary: Spectrally Adapting Regression for Covariate Shift
Benjamin Eyre (Columbia University), Richard Zemel (University of Toronto)
CodeDomain AdaptationImageTabular
π― What it does: This paper proposes a post-processing method for spectral adaptation of regression models under covariate shift, called SpAR, which projects the weights of the last layer of a pre-trained regression model using unlabeled target data to reduce out-of-distribution (OOD) errors caused by spectral inflation.
Outlier-Efficient Hopfield Layers for Large Transformer-Based Models
Jerry Yao-Chieh Hu (Northwestern University), Han Liu (Northwestern University)
CodeAnomaly DetectionOptimizationTransformerLarge Language ModelImageTextTime Series
π― What it does: A model based on the modern Hopfield network called 'Outlier-Efficient' is proposed, which improves the Transformer attention mechanism to reduce output anomalies caused by low-information words (such as punctuation and separators).
π― What it does: In a serverless decentralized federated learning scenario, this paper proposes the use of synthetic anchor data through feature regularization and knowledge distillation methods to alleviate data and model heterogeneity, thereby improving the generalization performance of local models on cross-domain tasks.
Overcoming Saturation in Density Ratio Estimation by Iterated Regularization
Lukas Gruber (Johannes Kepler University Linz), Werner Zellinger (Austrian Academy of Sciences)
CodeDomain AdaptationOptimizationImageText
π― What it does: Proposes and implements an iterative regularization method (iterative Tikhonov regularization) for density ratio estimation, overcoming the saturation phenomenon of traditional kernel methods and achieving faster error convergence across various algorithms.
PairNet: Training with Observed Pairs to Estimate Individual Treatment Effect
Lokesh Nagalapatti (Indian Institute of Technology Bombay), Sunita Sarawagi (Indian Institute of Technology Bombay)
CodeTabularBiomedical Data
π― What it does: A new Individual Treatment Effect (ITE) estimation training strategy called PairNet is proposed, which minimizes loss using only observed instance pairs;
π― What it does: For agent modeling of process systems, a physics-aware agent model (PAPM) is proposed that globally incorporates partial prior physical knowledge (general forms of various initial/boundary conditions and conservation relationships), along with a spatiotemporal stepping module (TSSM) designed to flexibly adapt to different process systems.
π― What it does: A parameter-efficient fine-tuning method based on Givens rotation (qGOFT) is proposed, which adapts large pre-trained models to downstream tasks through reparameterization while preserving the angular information of the pre-trained model.
Parameter Estimation in DAGs from Incomplete Data via Optimal Transport
Vy Vo (Monash University), Dinh Phung (VinAI Research)
CodeOptimizationImageGraphTime Series
π― What it does: In the context of missing data, this paper proposes a parameter learning framework based on optimal transport (OT) called OTP-DAG, aimed at estimating the parameters of probabilistic graphical models for arbitrary DAG structures.
PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition
Ziyang Zhang (University of Oxford), Jakob Nicolaus Foerster
CodeAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
π― What it does: The PARDEN method is proposed, which utilizes the LLM itself to first generate outputs, then has the LLM repeat those outputs, and uses BLEU similarity to determine if it is a jailbreak, thereby achieving defense.
π― What it does: A particle sampling framework PGPS based on Log-weighted Shrinkage (LwS) density path guidance is proposed, which uses neural networks to learn vector fields, allowing particles to migrate along a predetermined path from the initial distribution to the target posterior distribution.
π― What it does: For high-dimensional perceptual input, the PCLAST method is proposed to learn a plannable continuous latent state representation, and based on this, achieve multi-level planning.
π― What it does: This paper proposes a zero-shot cross-task preference alignment and robust reward learning framework (PEARL), which can utilize preference labels from the source task to generate pseudo-labels in the target task and learn policies.
PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning
Hyeong Kyu Choi (University of Wisconsin Madison), Yixuan Li (University of Wisconsin Madison)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A method called PICLe is proposed, which uses Bayesian inference and likelihood ratios to select examples in order to induce large language models to exhibit target personality traits through contextual learning.
PID: Prompt-Independent Data Protection Against Latent Diffusion Models
Ang Li (Peking University), Yisen Wang (Peking University)
CodeSafty and PrivacyAdversarial AttackDiffusion modelImage
π― What it does: This paper addresses the potential threats of privacy leakage and proposes a protection method for Latent Diffusion Models (PID) that does not rely on text prompts.
Piecewise Constant and Linear Regression Trees: An Optimal Dynamic Programming Approach
Mim van den Bos (Delft University of Technology), Emir DemiroviΔ (Delft University of Technology)
CodeOptimizationComputational EfficiencyTabular
π― What it does: Three optimal regression tree algorithms based on dynamic programming have been developed, namely piecewise constant regression trees, univariate linear regression trees, and multivariate linear regression trees, along with a dedicated acceleration algorithm for deep binary trees.
Planning, Fast and Slow: Online Reinforcement Learning with Action-Free Offline Data via Multiscale Planners
Chengjie Wu (Tsinghua University), Chongjie Zhang (Washington University in St. Louis)
CodeReinforcement LearningVideo
π― What it does: This paper investigates how to achieve efficient online reinforcement learning through a pre-trained value function and a two-level state-centered planner (with fast and slow scales) using passive dataset video data without action annotations.
π― What it does: The Stochastic deNOising REgularization (SNORE) framework is proposed, which modifies the PnP denoising step to denoise on noisy images and solves the inverse problem using stochastic gradient descent.
Position: A Call to Action for a Human-Centered AutoML Paradigm
Marius Lindauer (Leibniz University Hannover), Bernd Bischl (Ludwig Maximilians University Munich)
CodeLarge Language ModelReview/Survey Paper
π― What it does: This paper reviews the development and current status of Automated Machine Learning (AutoML), pointing out its shortcomings in transparency, customizability, interactivity, collaboration, and user empowerment, and calls for the establishment of a human-centered AutoML paradigm.
π― What it does: A critical analysis of the current state of research on time series anomaly detection is conducted, proposing and implementing various simple baselines (such as PCA reconstruction error, 1-NN distance, L2 norm, and a single layer MLP/MLPMixer/Transformer/GCN-LSTM), and comparing them with existing deep learning methods under standard and modified evaluation metrics.
Position: Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination
Zhiyao Luo (University of Oxford), Tingting Zhu (University of Oxford)
CodeReinforcement LearningBiomedical DataElectronic Health Records
π― What it does: This paper systematically evaluates the performance of various offline reinforcement learning (RL) algorithms in dynamic treatment regimes (DTR) through 17,550 experiments on a sepsis dataset, focusing on the diversity of policy evaluation methods, reward design, and benchmark settings and their impact on the results.
Position: Standardization of Behavioral Use Clauses is Necessary for the Adoption of Responsible Licensing of AI
Daniel McDuff (University of Washington), Danish Contractor
Code
π― What it does: Analyze and quantify the adoption of AI licenses (including behavioral usage terms) in AI models and software repositories, assess the differences in their terms, and propose standardization and tooling suggestions.
Position: Towards Implicit Prompt For Text-To-Image Models
Yue Yang (Shanghai Jiao Tong University), Ping Luo (Hong Kong University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageTextMultimodalityBenchmark
π― What it does: Proposes the concept of implicit prompts, constructs the ImplicitBench benchmark, and conducts a systematic evaluation on six mainstream text-to-image models.
PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs
Charlie Hou (Carnegie Mellon University), Daniel Lazar (Meta)
CodeData SynthesisFederated LearningSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The PrE-Text method is proposed, which generates synthetic text through differential privacy in a federated learning environment, serving as a replacement for traditional on-device training to train small and large models.
Prediction-powered Generalization of Causal Inferences
Ilker Demirel (Massachusetts Institute of Technology), David Sontag (Massachusetts Institute of Technology)
CodeMachine LearningTabular
π― What it does: This paper studies how to combine limited randomized controlled trials (RCTs) with large-scale observational data to enhance the external validity of causal inference for the target population.
π― What it does: A multi-task offline pre-training framework named Premier-TACO is proposed, which learns general visual representations through time-action driven contrastive learning, thereby enhancing the efficiency of few-shot policy learning.
π― What it does: This paper proposes a novel Gaussian Process Latent Variable Model (GPLVM) - advised RFLVM, which prevents model collapse by learning projection noise and using a Spectral Mixture kernel combined with differentiable Random Fourier Features (RFF) to achieve fully differentiable variational inference.