International Conference on Machine Learning · 550 papers
Principled Preferential Bayesian Optimization
Wenjie Xu (EPFL), Colin Jones
CodeOptimizationTabular
🎯 What it does: An optimistic preference Bayesian optimization algorithm based on likelihood ratio confidence sets (POP-BO) is proposed, along with an upper bound on cumulative loss and convergence rate from an information theory perspective.
Abdellah El Mrini (École Polytechnique Fédérale de Lausanne), Aurélien Bellet (Inria)
CodeFederated LearningSafty and PrivacyAdversarial AttackGraphTabular
🎯 What it does: This paper studies how 'honest-curious' attackers in decentralized learning can reconstruct private data of non-neighboring nodes by analyzing the information flow in Gossip averaging and decentralized gradient descent (D-GD).
Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models
Hengyi Wang (Rutgers University), Hao Wang (Rutgers University)
CodeExplainability and InterpretabilityTransformerContrastive LearningImage
🎯 What it does: PACE is proposed, a variational Bayesian framework designed to generate trustworthy, stable, sparse, and multi-level concept-level explanations for Vision Transformers (ViT).
Probabilistic Constrained Reinforcement Learning with Formal Interpretability
YANRAN WANG, David Boyle (Imperial College London)
CodeOptimizationExplainability and InterpretabilityReinforcement Learning
🎯 What it does: A new adaptive Wasserstein variational optimization framework, AWaVO, is proposed to achieve probabilistic interpretability in constrained reinforcement learning. It implements interpretable policy updates and inference through adaptive sliced Wasserstein distance and distributed representation optimization.
🎯 What it does: A generative model based on stochastic interpolation and the Föllmer process is proposed to map the current state of the system to the conditional distribution of future states, achieving unbiased probability predictions.
🎯 What it does: This paper proposes the use of Gaussian processes to learn probabilistic subgoal representations to improve subgoal abstraction and planning in hierarchical reinforcement learning.
Luca Beurer-Kellner (ETH Zurich), Martin Vechev (ETH Zurich)
CodeGenerationOptimizationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes a framework called Prompt Sketching, which views the generation process of large language models (LLMs) as a sequence decoding problem segmented by templates, achieving more structured reasoning through this framework.
Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts
Zhi-Yi Chin (National Yang Ming Chiao Tung University), Wei-Chen Chiu (National Yang Ming Chiao Tung University)
CodeGenerationSafty and PrivacyAdversarial AttackTransformerPrompt EngineeringDiffusion modelImageText
🎯 What it does: This paper proposes and implements Prompting4Debugging (P4D) — an automated red team tool designed to find harmful prompts that can bypass security mechanisms in text-to-image diffusion models.
Provably Better Explanations with Optimized Aggregation of Feature Attributions
Thomas Decker (Siemens AG), Florian Buettner (German Cancer Research Center)
CodeOptimizationExplainability and InterpretabilityConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a technique to enhance the quality of explanations by convexly combining multiple attribution results for interpretable models.
Pursuing Overall Welfare in Federated Learning through Sequential Decision Making
Seok-Ju Hahn (Ulsan National Institute of Science and Technology), Junghye Lee (Seoul National University)
CodeOptimizationFederated LearningImageTabular
🎯 What it does: The AAggFF framework is proposed, treating fair aggregation as an online convex optimization (OCO) problem. AAggFF-S (based on ONS) and AAggFF-D (based on EG and double robust estimation) are designed for cross-silo and cross-device scenarios, respectively, to enhance client-level fairness in federated learning (FL).
🎯 What it does: The study investigates how to efficiently train Quality-Diversity (QD) algorithms under resource constraints and proposes and implements the RefQD method.
🎯 What it does: This paper proposes an implicit neural representation network QIREN based on quantum data re-uploading circuit for efficient modeling of continuous signals.
QuRating: Selecting High-Quality Data for Training Language Models
Alexander Wettig (Princeton University), Danqi Chen (Princeton University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: By comparing text pairs using GPT-3.5, a scoring model is learned and these scores are used to select 30B training samples from the 260B token SlimPajama corpus, followed by training a 1.3B parameter language model;
R2E: Turning any Github Repository into a Programming Agent Environment
Naman Jain (University of California), Ion Stoica (University of California)
CodeAI Code AssistantTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes a framework called R2E that converts any GitHub repository into an interactive programming agent testing environment, and based on this, constructs the first large-scale real environment evaluation benchmark R2E-Eval1.
🎯 What it does: This paper studies a minimalist Random Masking method for Parameter-Efficient Fine-Tuning (PEFT) and demonstrates its competitive performance with traditional methods like LoRA across various language and vision tasks.
Rapid Learning without Catastrophic Forgetting in the Morris Water Maze
Raymond Wang, Ila R Fiete (Massachusetts Institute of Technology)
CodeConvolutional Neural NetworkSequential
🎯 What it does: A continuous and rapid learning framework based on biological principles (vHSN) has been constructed, capable of quickly learning new platform locations in a series of continuous Morris Water Maze (sWM) environments while retaining memory of old environments, achieving zero-forgetting transfer from new to old environments.
🎯 What it does: A UV map-based physical adversarial camouflage attack framework RAUCA is proposed, which evades vehicle detectors by optimizing the surface texture of vehicles.
🎯 What it does: A recursive early exit mechanism named ReeFL is proposed, which utilizes a shared Transformer-based module to fuse features from different depth sub-models, supporting multi-level early stopping and feature modulation on heterogeneous clients.
Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion
Xuantong LIU, Yuan Yao (Hong Kong University of Science and Technology)
CodeGenerationData SynthesisTransformerVision Language ModelScore-based ModelAuto EncoderImageText
🎯 What it does: A training-free text-to-image generation method based on a reverse Vision-Language Model (VLM) is proposed, which directly optimizes the image (latent) during the generation process to meet the text prompt.
Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints
Xiaobo Xia (University of Sydney), Tongliang Liu (University of Sydney)
CodeOptimizationImage
🎯 What it does: The refined core subset selection (RCS) problem is proposed, and a dictionary-based bi-level optimization algorithm is designed to minimize the size of the core subset while satisfying model performance constraints.
🎯 What it does: A framework for learning continuous normalizing flows (CNF) within constrained domains is proposed, called the Reflective Flow Matching (RFM) framework.
ReGAL: Refactoring Programs to Discover Generalizable Abstractions
Elias Stengel-Eskin (University of North Carolina), Mohit Bansal (University of North Carolina)
CodeAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: By refactoring and validating existing programs, an automatically learned reusable function library (REGAL) enhances the accuracy of large language models (LLMs) in program generation tasks.
🎯 What it does: Transformed from iGPT to D-iGPT by changing the prediction target from original pixels to semantic tokens provided by CLIP, and adding supervision on visible tokens, a stronger visual representation learning framework has been constructed.
🎯 What it does: A new relational deep neural network (DNN) verification framework called RACoon is proposed, which can accurately compute linear approximations by leveraging cross-execution dependencies across multiple executions and perform relational verification through MILP.
Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise
Thomas Pouplin (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
CodeTabular
🎯 What it does: This paper studies an algorithm for directly learning prediction intervals—Relaxed Quantile Regression (RQR), which eliminates the limitations of traditional quantile assumptions and can generate more flexible interval predictions.
ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages
Andrew Jesson (University of Oxford), Yarin Gal (University of Oxford)
CodeReinforcement Learning
🎯 What it does: Three modifications were introduced based on A3C: ReLU truncation of advantage estimates (only updating positive advantages), spectral normalization weights for both actor and critic networks, and implementing approximate Bayesian inference through Dropout, resulting in positive advantage updates, stable value estimates, and adaptive exploration.
🎯 What it does: This paper constructs the BW-ReLU network by adding constraints based on second-order B-spline wavelets to ReLU neurons, achieving the learning of implicit neural representations (INR).
Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimation
Floris Holstege (University of Amsterdam), Cees Diks (University of Amsterdam)
CodeRepresentation LearningText
🎯 What it does: A post-hoc concept removal method is proposed—Joint Subspace Estimation (JSE), which simultaneously identifies the orthogonal low-dimensional subspaces of the main task and pseudo-concepts to eliminate pseudo-concepts from neural network embeddings without losing information about the main task.
🎯 What it does: A self-triggering retrieval-enhanced generation framework called REPOFORMER is proposed, which can determine whether to retrieve cross-file information based on context in repository-level code completion, avoiding ineffective or harmful retrievals.
Representation Surgery: Theory and Practice of Affine Steering
Shashwat Singh (International Institute of Information Technology Hyderabad), Ponnurangam Kumaraguru (International Institute of Information Technology Hyderabad)
CodeGenerationOptimizationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Two optimal affine steering functions (mean matching and mean + covariance matching) are proposed to guide language models to produce fairer and non-toxic content without changing the internal representations of the model.
Reservoir Computing for Short High-Dimensional Time Series: an Application to SARS-CoV-2 Hospitalization Forecast
Thomas Ferté (Bordeaux University), Xavier Hinaut (Bordeaux University)
CodeOptimizationHyperparameter SearchTime SeriesSequentialBiomedical DataElectronic Health Records
🎯 What it does: Using Reservoir Computing (RC) combined with Genetic Algorithm (GA) to perform a 14-day short-term prediction of SARS-CoV-2 hospitalizations at the University Hospital of Bordeaux, with training data consisting of a high-dimensional time series of 586 days and 409 features.
Rethinking DP-SGD in Discrete Domain: Exploring Logistic Distribution in the Realm of signSGD
Jonggyu Jang (Pohang University of Science and Technology), Hyun Jong Yang (Pohang University of Science and Technology)
CodeOptimizationFederated LearningSafty and PrivacyConvolutional Neural NetworkTransformerImage
🎯 What it does: In response to DP-SIGNSGD in federated/distributed training, the authors propose using Logistic noise instead of traditional Gaussian noise, and provide corresponding theoretical proofs and algorithm implementation (DP-SIGNLOSGD).
🎯 What it does: A joint-cluster supervised learning framework is proposed, improving the traditional node-independent cross-entropy loss by considering the joint distribution of graph nodes and clusters to enhance classification accuracy and robustness.
🎯 What it does: This paper explores the effectiveness of Transformers in partially observable Markov decision processes (POMDPs) and theoretically and experimentally demonstrates their fundamental limitations in learning tasks related to regular languages, proposing Linear Recursive Units (LRU) as a more suitable sequence model.
🎯 What it does: A scalable Hessian diagonal approximation method called HesScale is proposed, and it is applied to the second-order optimizer AdaHesScale in supervised learning and a step size scaling scheme in reinforcement learning.
🎯 What it does: Based on the pre-trained visual Transformer, a Self-Prompt Tuning (SPT) scheme is proposed, initializing the embedding prototypes of downstream tasks as learnable prompt words, and only updating the prompt words and task head during the training process.
Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
Rui Yang (Tencent AI Lab), Jianshu Chen (Tencent AI Lab)
CodeRecommendation SystemOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningImageText
🎯 What it does: The Rewards-in-Context (RiC) method is proposed, which achieves multi-objective alignment of large models through supervised fine-tuning (SFT) by embedding multiple reward information in the prompts, and dynamically adjusts rewards during the inference phase to meet user preferences.
🎯 What it does: The RMIB framework is proposed, which improves the performance of text matching tasks by aligning the prior distributions of heterogeneous domain text representations and combining information bottleneck theory.
Mehran Poursoltani (McGill University), Angelos Georghiou (University of Cyprus)
CodeOptimizationTabular
🎯 What it does: A distributionally robust context optimization framework is designed and solved, directly maximizing the 'prescriptiveness coefficient' to obtain decision strategies that better utilize side information.
Taha Ameen (University of Illinois Urbana-Champaign), Bruce Hajek (University of Illinois Urbana-Champaign)
CodeGraph Neural NetworkGraph
🎯 What it does: The paper studies the graph matching problem in the presence of tampered nodes, proposing two models: weak adversary (random tampering) and strong adversary (observable and arbitrary tampering), and provides the infeasibility, feasibility, and accuracy thresholds for matching.
🎯 What it does: This paper proposes a task weight update method based on 'excess risk' called ExcessMTL, aimed at addressing the imbalance in weight allocation caused by label noise in multi-task learning.
Robust Optimization in Protein Fitness Landscapes Using Reinforcement Learning in Latent Space
Minji Lee (Korea Advanced Institute of Science and Technology), Ho Min Kim (Korea Advanced Institute of Science and Technology)
CodeOptimizationReinforcement LearningPrompt EngineeringBiomedical Data
🎯 What it does: Under the starting point of low fitness protein sequences, a reinforcement learning-based framework called LatProtRL is proposed, which utilizes a pre-trained protein language model to learn latent representations and conducts multi-step exploration in the latent space to optimize protein function (fitness).
🎯 What it does: By integrating spectral domain filtering, a fast MLP encoder, and spatial positive sample pairs within a contrastive learning framework, the S3GCL method is proposed for self-supervised learning of node representations on graphs with varying levels of homogeneity.
🎯 What it does: This paper proposes a dual-stage method called S3O, which can simultaneously learn visible 3D shapes, variable poses, and underlying skeletal structures using only a single unlabelled monocular video, without the need for preset models, camera poses, or key points.
SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention
Romain Ilbert (Huawei Noah's Ark Lab), Ievgen Redko
CodeTransformerTime Series
🎯 What it does: Proposes the SAMformer model to address the underfitting and training instability issues of Transformer in multivariate long-term time series forecasting.
Aimee Maurais (Massachusetts Institute of Technology), Youssef Marzouk (Massachusetts Institute of Technology)
CodeFlow-based ModelMultimodality
🎯 What it does: A gradient-independent sampling framework based on Fisher-Rao gradient flow is proposed—Kernel Fisher-Rao Flow (KFRFlow), which can transfer the reference distribution to the target distribution within a unit time.
CodeClassificationRecognitionGenerationTransformerLarge Language ModelImage
🎯 What it does: Proposed and trained a large-scale autoregressive image model (AIM), transferring the autoregressive pre-training approach of large language models to the visual domain;
Scalable Safe Policy Improvement for Factored Multi-Agent MDPs
Federico Bianchi (University of Verona), Alessandro Farinelli (University of Verona)
CodeOptimizationReinforcement LearningBenchmark
🎯 What it does: A scalable safe policy improvement algorithm for factored multi-agent Markov decision processes (FV-MCTS-SPIBB) is proposed, which generates safe improved policies by using the constrained Max-Plus/Var-El method in MCTS.
🎯 What it does: Designed and implemented a keypoint detector MorseDet based on differentiable persistent homology, proposing an unsupervised loss function centered on persistence and boundary similarity, achieving scale-invariant and learnable keypoint detection.
SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code
Ziniu Hu (California Institute of Technology), Alireza Fathi (Google DeepMind)
CodeGenerationData SynthesisOptimizationAI Code AssistantTransformerLarge Language ModelTextMeshRetrieval-Augmented Generation
🎯 What it does: Developed SceneCraft, an LLM-based Blender code generator that converts natural language descriptions into executable 3D scene scripts.
🎯 What it does: Distill the pre-trained diffusion model into a single-step generator, achieving exponential FID reduction using a data-free distillation method.
🎯 What it does: By constructing a dual-branch Kernel-Eigen Pair Sparse Variational Gaussian Process (KEP-SVGP) model, Bayesian uncertainty estimation of Transformer self-attention is achieved.
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
Zixiang Chen (University of California), Quanquan Gu (University of California)
CodeGenerationData SynthesisReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Proposes a Self-Play Fine-tuning (SPIN) method that iteratively improves model performance using synthetic data generated by a weak LLM without the need for additional human annotations or external rewards.
Self-Supervised Coarsening of Unstructured Grid with Automatic Differentiation
Sergei Shumilin (Skolkovo Institute of Science and Technology), Vladimir Vanovskiy (Skolkovo Institute of Science and Technology)
CodeOptimizationComputational EfficiencyGraph Neural NetworkPoint CloudPhysics Related
🎯 What it does: A self-supervised coarsening algorithm based on differentiable physics for unstructured grids is proposed, utilizing k-means, automatic differentiation, and stochastic minimization to reduce the number of grid points while maintaining numerical simulation accuracy.
SelMatch: Effectively Scaling Up Dataset Distillation via Selection-Based Initialization and Partial Updates by Trajectory Matching
Yongmin Lee (Korea Advanced Institute of Science and Technology), Hye Won Chung (Korea Advanced Institute of Science and Technology)
CodeKnowledge DistillationImage
🎯 What it does: A dataset distillation method named SelMatch is proposed, which effectively expands the distillation scale through selective initialization and partial updates.
🎯 What it does: By guiding the diffusion model with pose quality and interaction boundary information, the SA-HOI framework is proposed, which can generate higher quality person-object interaction images from pure text prompts and achieve multi-step refinement based on iterative inversion.
Semantically-correlated memories in a dense associative model
Thomas F Burns (Brown University)
CodeGraph Neural NetworkVideoGraph
🎯 What it does: The Correlated Dense Associative Memory (CDAM) model is proposed, which unifies automatic association and disassociation into a dense associative memory network, and achieves semantic association through a graph structure. The four dynamic patterns are then analyzed theoretically and numerically, demonstrating applications in tasks such as community detection, sequence recall, and finite automaton simulation.
Sign is Not a Remedy: Multiset-to-Multiset Message Passing for Learning on Heterophilic Graphs
Langzhang Liang (Harbin Institute of Technology), Yuan Qi (Fudan University)
CodeClassificationGraph Neural NetworkGraph
🎯 What it does: This paper addresses the Signed Message Passing (SMP) method used on heterophilic graphs, revealing its two major flaws from both theoretical and experimental perspectives: (1) even if the first-order propagation weights are 'desirable', the multi-hop propagation matrix may become undesirable, leading to incorrect updates of node embeddings; (2) even with negative weights, SMP is still prone to the oversmoothing problem in multi-class graphs. To address these issues, the authors propose a new Multiset-to-Multiset (m-2-m) message passing framework and design the M2M-GNN model based on it. M2M-GNN partitions neighbor embeddings into several 'chunks' using attention soft labels, aggregates and concatenates each chunk separately, thereby maintaining the separation of different class embeddings and enhancing expressive power. Extensive experiments on 11 benchmark datasets demonstrate that M2M-GNN ranks first overall in node classification tasks, achieving at least a top-three performance on each dataset and maintaining robustness in deep models.
Simple linear attention language models balance the recall-throughput tradeoff
Simran Arora (Stanford University), Christopher Re (Stanford University)
CodeGenerationRetrievalTransformerLarge Language ModelText
🎯 What it does: A hybrid language model architecture named BASED is proposed, which combines global linear attention (Taylor approximation) with local sliding window attention to achieve a trade-off between high recall rate and high throughput.
Simulation-Based Inference with Quantile Regression
He Jia (Princeton University)
CodeRecurrent Neural NetworkTime SeriesBenchmark
🎯 What it does: This paper proposes Neural Quantile Estimation (NQE), a method that utilizes conditional quantile regression for autoregressive learning of univariate quantiles for each posterior dimension, and obtains high-quality posterior samples through quantile interpolation and post-processing calibration.
SIN: Selective and Interpretable Normalization for Long-Term Time Series Forecasting
Lu Han (Nanjing University), De-Chuan Zhan (Nanjing University)
CodeExplainability and InterpretabilityComputational EfficiencyTransformerTime SeriesFinance Related
🎯 What it does: A learnable selective interpretable normalization method for long sequence prediction (SIN) is proposed, which normalizes and denormalizes by learning statistics to alleviate distribution drift caused by non-stationarity.
🎯 What it does: This paper proposes a Transformer structure that gradually replaces LayerNorm with re-parameterized BatchNorm and introduces simplified linear attention to reduce computational overhead during inference.
🎯 What it does: A multi-modal sleep foundation model, SleepFM, was constructed, utilizing contrastive learning to learn unified representations from EEG, ECG, and respiratory signals.
Sliding Down the Stairs: How Correlated Latent Variables Accelerate Learning with Neural Networks
Lorenzo Bardone (International School of Advanced Studies), Sebastian Goldt (International School of Advanced Studies)
CodeImage
🎯 What it does: This paper studies how neural networks can effectively extract information from high-order input cumulants, particularly by leveraging the correlations between latent variables to accelerate learning.
🎯 What it does: Developed Slot Abstractor, which integrates Slot Attention and Abstractor modules for scalable abstract visual reasoning, capable of handling tasks with a large number of objects and multiple relationships.
🎯 What it does: Proposes SMaRt - incorporating score matching regularization from a pre-trained diffusion model into GAN training to address the gradient vanishing problem.
Haotian Lin (Pennsylvania State University), Matthew Reimherr (Pennsylvania State University)
CodeDomain AdaptationOptimizationTabular
🎯 What it does: This paper proposes Smoothness Adaptive Transfer Learning (SATL), an algorithm that adaptively estimates the smoothness of the target and source models, as well as the shift function, using a Gaussian kernel in a two-stage shift transfer learning framework.
Mark Kozdoba (Technion Israel Institute of Technology), Shie Mannor (NVIDIA Research)
CodeAnomaly DetectionTabular
🎯 What it does: A non-parametric pre-density estimation method SOSREP based on Sobolev norm regularization is proposed to address the problem of high-dimensional unnormalized density estimation.
Socialized Learning: Making Each Other Better Through Multi-Agent Collaboration
Xinjie Yao (Tianjin University), Qinghua Hu (Tianjin University)
CodeFederated LearningKnowledge DistillationImage
🎯 What it does: A multi-agent social learning (MASC) framework is proposed, which achieves knowledge transfer from expert to general categories through two main modules: collective cooperation and mutual benefit.
Sparse is Enough in Fine-tuning Pre-trained Large Language Models
Weixi Song (Wuhan University), Bo Du (Wuhan University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a novel sparse fine-tuning method called SIFT and analyzes the generalization advantages of pre-trained models compared to training from scratch through PAC-Bayes theory, further explaining the quasi-sparsity characteristics of gradients and intrinsic dimensionality compression. Based on this, the authors implement a sparse fine-tuning strategy that updates only a small number of parameters and validate its effectiveness on GLUE and instruction fine-tuning tasks.
🎯 What it does: For large Vision Transformer (ViT) models, a Sparse Model Inversion (SMI) method is proposed, which efficiently achieves data-free applications such as model quantization and knowledge transfer by only recovering semantic foreground areas and stopping the inversion of uninformative backgrounds.
🎯 What it does: A sparse-dense multimodal image registration method SDME based on a multi-task network is proposed, which first uses sparse matching for initialization and then refines with dense alignment.
🎯 What it does: This work proposes the DIFFS4L framework, which utilizes diffusion models to generate diverse synthetic speech (including new prosody, new speakers, and meaningless babble) on low-resource unlabeled speech data, thereby expanding the pre-training corpus and enhancing self-supervised speech representation learning.
SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
Xingrun Xing (Institute of Automation, Chinese Academy of Sciences), Guoqi Li (Institute of Automation, Chinese Academy of Sciences)
CodeSpiking Neural NetworkTransformerLarge Language ModelText
🎯 What it does: A completely pulse-based universal language model, SpikeLM, is proposed, utilizing elastic bidirectional pulse coding to achieve end-to-end pulse computation for language tasks.
🎯 What it does: A method for converting ANN to SNN based on Transformer, called SpikeZIP-TF, is proposed, achieving strict equivalence between quantized Transformer ANN and SNN without loss of accuracy after conversion.
Split-and-Denoise: Protect large language model inference with local differential privacy
Peihua Mai (National University of Singapore), Yan Pang (National University of Singapore)
CodeOptimizationSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: This paper proposes the Split-N-Denoise (SnD) framework, which splits large language models into a local token embedding layer and a cloud-based subsequent layer. Users add noise to the embeddings locally before uploading them, and the cloud returns outputs affected by the noise. Users then perform denoising locally using the known noise to ultimately obtain high-quality embeddings for downstream tasks.
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Abhimanyu Hans (University of Maryland), Tom Goldstein (University of Maryland)
CodeClassificationGenerationTransformerLarge Language ModelText
🎯 What it does: A zero-shot LLM text detection method called Binoculars is proposed, which uses the cross perplexity ratio of two similar LLMs to distinguish between human and machine-generated text.
Sehoon Kim (University of California Berkeley), Kurt Keutzer (University of California Berkeley)
CodeGenerationCompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A post-training quantization framework called SqueezeLLM is proposed for large language models, which can maintain nearly the same generation performance as FP16 at extremely low precision while significantly improving inference speed.
🎯 What it does: This paper studies the impact of the stability region of numerical solvers on the training and prediction performance of Neural ODEs, and proposes a stability-informed initialization method (SII) based on the stability region, significantly improving training efficiency and model performance.
Standardized Interpretable Fairness Measures for Continuous Risk Scores
Ann-Kristin Becker (SCHUFA Holding AG), Klaus Broelemann (SCHUFA Holding AG)
CodeExplainability and InterpretabilityTabular
🎯 What it does: A standardized and interpretable fairness metric method for continuous risk scores is proposed, which quantifies inequality between groups based on the Wasserstein-1 distance.
Statistical Inference Under Constrained Selection Bias
Santiago Cortes-Gomez (Carnegie Mellon University), Bryan Wilder (Carnegie Mellon University)
CodeOptimizationTabular
🎯 What it does: A framework is proposed that utilizes known target distribution expectation constraints to partially identify statistics in the presence of selection bias and provides high-probability interval estimates.
🎯 What it does: A Stereographic Spherical Sliced Wasserstein (S3W) distance based on spherical projection and generalized Radon transform is proposed for efficient comparison of spherical probability distributions.
🎯 What it does: A random conditional diffusion model (SCDM) based on label diffusion is proposed to improve the robustness of semantic image synthesis under noisy labels.
🎯 What it does: A generative model using data-dependent coupling within a random interpolation framework is proposed, which can pair the base distribution with the target distribution through conditional coupling, thereby obtaining a more suitable transport path in continuous time mapping.
🎯 What it does: In the self-supervised Masked Image Modeling (MIM) framework, Stochastic Positional Embeddings (StoP) are introduced to allow the model to consider positional uncertainty when predicting masked patches, thereby improving representation learning.
🎯 What it does: A method is proposed to enhance the sub-token embedding resolution of Vision Transformer by applying translational perturbations to the input images and aggregating at the feature level, without training or modifying the model.
🎯 What it does: This paper presents an analysis of the existence and uniqueness of fixed points in deep equilibrium networks (DEQ) by combining sub-homogeneous operator theory with nonlinear Perron-Frobenius theory, and based on this, designs a sub-homogeneous deep equilibrium model (SubDEQ).
Submodular framework for structured-sparse optimal transport
Piyushi Manupriya (Indian Institute of Technology Hyderabad), Bamdev Mishra (Microsoft)
CodeOptimizationMixture of ExpertsTabular
🎯 What it does: This paper proposes a structured sparse unbalanced optimal transport (UOT) framework that can learn interpretable sparse transport plans while satisfying upper bound sparsity constraints.
🎯 What it does: A performance prediction method based on Graph Neural Features (GRAF) is proposed, which can accurately predict the performance of architectures in the NAS search space without training the network.
Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimization
Hyeonah Kim (Korea Advanced Institute of Science and Technology), Jinkyoo Park
CodeOptimizationReinforcement Learning
🎯 What it does: This paper proposes a Symmetric Replay Training (SRT) method, which enhances the sample efficiency of deep reinforcement learning in combinatorial optimization by performing symmetric transformations on high-reward trajectories obtained and replaying them without additional reward evaluations.
🎯 What it does: A dual GAT structure is proposed to address the non-stationarity problem in reinforcement learning by learning 'Causal Origin Representation (COREP)'.
Tackling Prevalent Conditions in Unsupervised Combinatorial Optimization: Cardinality, Minimum, Covering, and More
Fanchen Bu (Korea Advanced Institute of Science and Technology), Kijung Shin (Korea Advanced Institute of Science and Technology)
CodeOptimizationGraph Neural NetworkGraph
🎯 What it does: The UCOM2 framework is proposed for unsupervised solving of combinatorial optimization problems, within which probability objectives are constructed for common constraints (cardinality constraints, minimum/maximum subset distance, coverage, clique/independent set, non-binary decisions, and uncertainty) and an incremental greedy de-randomization algorithm is designed.
CodeDrug DiscoveryTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Utilizing a general large language model, domain labels and functional labels that can be learned are introduced for task solving in specialized fields (such as multilingual translation, protein sequences, and chemical SMILES).
🎯 What it does: Proposes Taylor Video, a method that extracts motion information of displacement, velocity, and acceleration in three channels from the differences in grayscale frames using Taylor expansion;