International Conference on Machine Learning Β· 550 papers
Compositional Few-Shot Class-Incremental Learning
Yixiong Zou (Huazhong University of Science and Technology), Ruixuan Li (Huazhong University of Science and Technology)
CodeClassificationExplainability and InterpretabilityConvolutional Neural NetworkSupervised Fine-TuningImage
π― What it does: This paper proposes a cognitive-inspired compositional method that uses raw image patches as transferable 'primitives' and completes few-shot class incremental learning classification through the similarity of the primitive set; at the same time, a primitive reuse module is designed to enhance cross-class sharing, avoid forgetting, and improve interpretability.
Compressing Large Language Models by Joint Sparsification and Quantization
Jinyang Guo (Beihang University), Xianglong Liu (Beihang University)
CodeCompressionTransformerLarge Language ModelText
π― What it does: A Joint Sparsification and Quantization (JSQ) framework is proposed for high compression of large-scale language models, eliminating useless outliers through an activation editor before sparsification.
Xiao Zhang (Tsinghua University), Ji Wu (Tsinghua University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: By appending context in front of the document during fine-tuning, a context-conditioned language model is learned, enabling selective learning of knowledge.
Conditionally-Conjugate Gaussian Process Factor Analysis for Spike Count Data via Data Augmentation
Yididiya Y. Nadew (Iowa State University), Christopher John Quinn
CodeTime SeriesSequentialBiomedical Data
π― What it does: A conditional conjugate Gaussian process factor analysis (ccGPFA) model is proposed to handle neural spike count data, avoiding the inference difficulties caused by traditional non-conjugate likelihoods;
π― What it does: This paper proposes Sorted Adaptive Prediction Sets (SAPS), a non-conformity scoring method in conformal prediction of deep classifiers that retains only the maximum softmax probability while discarding the remaining probability values.
Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration
Shi-ang Qi (University of Alberta), Russell Greiner (University of Alberta)
CodeBiomedical DataElectronic Health Records
π― What it does: A general post-processing framework called CSD is proposed to enhance the calibration of survival models while maintaining unchanged discriminative performance.
Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentations
Helen Qu (University of Pennsylvania), Sang Michael Xie (Stanford University)
CodeDomain AdaptationContrastive LearningTime Series
π― What it does: A Fine-Tuning framework called Connect Later is proposed, which uses targeted data augmentation after pre-training to enhance the model's robustness in the target domain.
Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models
Zhengbo Wang (University of Science and Technology of China), Tieniu Tan (Institute of Automation, Chinese Academy of Sciences)
CodeClassificationDomain AdaptationOptimizationTransformerPrompt EngineeringVision Language ModelImage
π― What it does: A collaborative fine-tuning framework called CraFT is proposed for black-box vision-language models (VLM), which can adapt to downstream tasks solely through input prompts and output predictions.
Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data
Giannis Daras (University of Texas at Austin), Constantinos Costis Daskalakis (Massachusetts Institute of Technology)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: A complete framework based on a dual Tweedie formula and consistency loss is proposed, which can learn a diffusion model that is completely consistent with the original distribution using training samples that only contain noise.
Paul Duetting, Morteza Zadimoghaddam (Google Research)
CodeOptimizationGraph
π― What it does: A submodular function maximization algorithm that maintains consistency in a streaming environment is proposed, which can change only a limited number of elements with each insertion while maintaining an approximately optimal solution.
Constrained Ensemble Exploration for Unsupervised Skill Discovery
Chenjia Bai (Shanghai Artificial Intelligence Laboratory), Xuelong Li (Shanghai Artificial Intelligence Laboratory)
CodeRobotic IntelligenceReinforcement Learning
π― What it does: An unsupervised skill learning framework called CeSD is proposed, which can automatically discover diverse and comprehensive skills without using external rewards.
π― What it does: A mutual information (MI) gap measurement method based on contrastive learning is proposed, utilizing the MI gap of source domain data to filter and select the most useful transition data, thereby improving the sample efficiency of cross-domain offline reinforcement learning.
π― What it does: This paper proposes an unsupervised method called convSeq, which utilizes backpropagation to optimize 2D filters for the rapid detection of spatiotemporal patterns in neural spike data.
Counterfactual Metarules for Local and Global Recourse
Tom Bewley (J.P. Morgan), Manuela Veloso (J.P. Morgan)
CodeOptimizationExplainability and InterpretabilityComputational EfficiencyTabular
π― What it does: A model-agnostic algorithm T-CREx is proposed, which learns interpretable counterfactual rules and their meta-rules using tree-based surrogate models, providing both individual-level counterfactuals and actionable global solutions.
π― What it does: A lightweight joint fine-tuning framework CRoFT is proposed, which can maintain the pre-trained knowledge of CLIP while addressing OOD generalization and open-set OOD detection.
π― What it does: A method named CROW is proposed for cross-domain open-world discovery tasks, which involves labeling samples with known categories and discovering unknown categories in the presence of domain shift and category shift.
π― What it does: A cross-domain strategy adaptation method based on representation mismatch is proposed, which quantifies the dynamic differences between the source domain and the target domain through state and state-action encoders trained in the target domain, using representation mismatch as a reward penalty to guide SAC learning.
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
Dachuan Shi (Shanghai AI Laboratory), Jiaqi Wang (Shanghai AI Laboratory)
CodeComputational EfficiencyTransformerVision Language ModelMultimodality
π― What it does: This paper proposes a cross-modal guided Token aggregation framework called CrossGET, which can dynamically merge redundant Tokens in a visual-language Transformer, significantly reducing computational load and improving inference speed.
Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes
Nabeel Seedat (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
CodeData SynthesisTransformerLarge Language ModelPrompt EngineeringTabular
π― What it does: Under low sample conditions (<100), high-quality data augmentation is achieved through the generation of tabular data using LLMs (such as GPT-4) combined with a learning dynamic-driven filtering mechanism.
π― What it does: This paper proposes and implements CurBench, a cross-domain curriculum learning benchmark covering 15 datasets, 9 benchmark models, and 3 training settings (standard, noisy, imbalanced) across computer vision, natural language processing, and graph structure learning;
Data Engineering for Scaling Language Models to 128K Context
Yao Fu (University of Edinburgh), Hao Peng (University of Illinois at Urbana-Champaign)
CodeTransformerLarge Language ModelText
π― What it does: By performing full attention continuous pre-training on 7B/13B LLaMA-2 with 1-5B tokens, the model's context window is expanded to 128K.
π― What it does: This paper proposes a data-efficient autoregressive visual model DeLVM, which achieves multi-task learning on limited data through data augmentation and knowledge distillation.
Data-free Neural Representation Compression with Riemannian Neural Dynamics
Zhengqi Pei (Institute of Computing Technology Chinese Academy of Sciences), Qingming Huang (School of Computer Science and Technology University of Chinese Academy of Sciences)
CodeCompressionComputational EfficiencyImage
π― What it does: Reconstruct the weight matrix of the pre-trained neural network under Riemannian metrics to achieve data-free compression and inference acceleration.
Lingxiao Li (Massachusetts Institute of Technology), Lester Mackey (Microsoft Research)
CodeCompressionOptimizationTabular
π― What it does: A series of distribution compression methods for biased sampling sequences are proposed, capable of generating high-quality, sparse representative point sets (coresets) to approximate the target distribution P.
π― What it does: This paper proposes a context encoder DORA that learns from offline data, capable of quickly identifying and adapting to non-stationary dynamic environments, and trains transferable meta-policies within an offline reinforcement learning framework.
Deconstructing the Goldilocks Zone of Neural Network Initialization
Artem M Vysogorets, Julia Kempe (New York University)
CodeOptimizationConvolutional Neural NetworkImage
π― What it does: Analyze and redefine the Goldilocks zone during neural network initialization, derive the essential conditions that lead to excessive positive curvature, and clarify its relationship with model confidence, initial loss, and gradient vanishing through theory and experiments.
CodeGenerationComputational EfficiencyTransformerLarge Language ModelText
π― What it does: A distributed LLM service system named D'ej'Vu has been constructed, utilizing the KV cache streaming library (D'ej'VuLib) to achieve the separation of prompting and generation phases, micro-batch exchange, and KV cache replication, thereby improving throughput, reducing GPU memory usage, and enabling fault tolerance.
π― What it does: This paper proposes a graph rewiring method based on Delaunay triangulation (Delaunay Rewiring, abbreviated as DR), which alleviates the issues of over-squashing and over-smoothing in GNNs by reconstructing the graph structure using only node features, achieving higher node classification accuracy.
π― What it does: This paper proposes Density-Softmax, a model that requires only one forward pass during inference, is sampling-free, and lightweight, aimed at improving uncertainty estimation and robustness under distribution shifts.
π― What it does: This paper proposes a knowledge distillation method called Disparity Feature Distillation (DFD), which separates spatial regions based on the feature response differences between the teacher model and the student model, applying different distillation constraints to regions with varying degrees of disparity.
π― What it does: Designed and implemented DFlow, a generative framework that combines denoising autoencoders and normalizing flows for high-fidelity speech waveform synthesis.
DiffAug: Enhance Unsupervised Contrastive Learning with Domain-Knowledge-Free Diffusion-based Data Augmentation
Zelin Zang (Westlake University), Yang You (National University of Singapore)
CodeRepresentation LearningData-Centric LearningDiffusion modelContrastive LearningImageBiomedical Data
π― What it does: This paper proposes DiffAug, an augmentation framework for unsupervised contrastive learning that generates conditional vectors through a semantic encoder and uses a conditional diffusion model to generate positive samples, enhancing representation learning effectiveness.
Xutao Ma (Shanghai Jiao Tong University), WenLi Du
CodeOptimizationReinforcement LearningTabularTime SeriesFinance Related
π― What it does: This paper proposes a differentiable distributionally robust optimization (DRO) layer and embeds it into a decision-focused learning pipeline, supporting mixed-integer decisions and parameterized second-order cone (SOC) uncertainty sets.
Differentiable Mapper for Topological Optimization of Data Representation
Ziyad Oulhaj (Nantes Universite), Bertrand Michel (Nantes Universite)
CodeOptimizationRepresentation LearningPoint CloudBiomedical Data
π― What it does: This paper introduces Soft Mapper, a differentiable probabilistic modification of the Mapper graph, which automatically optimizes the filter function using topological loss (such as total persistence) to obtain Mapper representations with richer topological information.
π― What it does: Proposes the Differentiable Weightless Neural Network (DWN), a multi-layer weightless neural network based on lookup tables (LUT), and achieves training through gradient descent;
Differentially Private Decentralized Learning with Random Walks
Edwige Cyffers (Universite de Lille), Jalaj Upadhyay (Rutgers University)
CodeOptimizationFederated LearningSafty and PrivacyGraph Neural NetworkGraphTabular
π― What it does: This paper proposes a private random walk-based stochastic gradient descent (RW-DP-SGD) algorithm for decentralized distributed learning, and provides convergence analysis under strongly convex functions and a closed-form estimate of privacy leakage for arbitrary graph structures.
Differentially Private Post-Processing for Fair Regression
Ruicheng Xian (University of Illinois Urbana-Champaign), Han Zhao (University of Illinois Urbana-Champaign)
CodeOptimizationSafty and PrivacyTabular
π― What it does: A post-processing algorithm based on differential privacy is proposed, which can adjust the fairness of the regressor without changing the training phase.
Differentially Private Synthetic Data via Foundation Model APIs 2: Text
Chulin Xie (University of Illinois Urbana-Champaign), Sergey Yekhanin (Microsoft Research)
CodeData SynthesisSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes an algorithm named AUG-PE, which generates differential privacy (DP) synthetic text by utilizing only the API of large language models (LLMs) without the need for model training.
DiNADO: Norm-Disentangled Neurally-Decomposed Oracles for Controlling Language Models
Sidi Lu (University of California), Nanyun Peng (University of California)
CodeGenerationTransformerLarge Language ModelReinforcement LearningText
π― What it does: This paper presents DiNADO, an improved Neurally-Decomposed Oracle (NADO) framework that enables controllable text generation by splitting global norms and stepwise reward values.
Discovering Features with Synergistic Interactions in Multiple Views
Chohee Kim (Chung Ang University), Changhee Lee
CodeBiomedical Data
π― What it does: This paper proposes a multi-view feature selection framework, SynFS, which can simultaneously discover feature subsets with both synergistic and non-synergistic interaction information, thereby capturing the interactions between different views more comprehensively.
π― What it does: This study investigates how to use steganography to hide copyrighted images in the training data of potential diffusion models (LDM) and demonstrates that these hidden images can lead to the model replicating copyrighted content during inference, thereby achieving indirect copyright infringement.
Disparate Impact on Group Accuracy of Linearization for Private Inference
Saswat Das (University of Virginia), Ferdinando Fioretto (University of Virginia)
CodeClassificationRecognitionSafty and PrivacyKnowledge DistillationConvolutional Neural NetworkImage
π― What it does: The study investigates the impact of ReLU linearization on model fairness when implementing privacy inference in neural networks and proposes a mitigation strategy that incorporates fairness regularization during the fine-tuning phase.
Distributed High-Dimensional Quantile Regression: Estimation Efficiency and Support Recovery
Caixing Wang (Shanghai University of Finance and Economics), Ziliang Shen (Shanghai University of Finance and Economics)
CodeOptimizationTabular
π― What it does: This study investigates the estimation efficiency and support recovery of distributed high-dimensional quantile regression, proposing a distributed algorithm based on double smoothing.
DITTO: Diffusion Inference-Time T-Optimization for Music Generation
Zachary Novack (University of California San Diego), Nicholas J. Bryan (Adobe Research)
CodeGenerationOptimizationDiffusion modelAudio
π― What it does: This paper presents DITTO, a method that achieves various fine-grained controls (such as illustration, extension, repetition, intensity, melody, and structure) for a pre-trained text-to-music diffusion model during the inference phase by optimizing the initial noise latent vector and using any differentiable feature matching loss without additional training.
π― What it does: A differentiable multi-task grouping (DMTG) method is proposed, which simultaneously completes task grouping and model learning during the training process;
π― What it does: This paper proposes an action-conditioned world model (AWM) that improves gradient-based policy optimization by mapping the world model solely as a sequence of actions.
π― What it does: This paper studies the role of label smoothing in deep partial label learning (PLL), proposes theoretical risk bounds, derives the optimal smoothing rate, and presents the LS-PLL algorithm based on this theory.
Anurag Singh (Saarland University), Krikamol Muandet (University of Oxford)
CodeDomain AdaptationOptimizationTabular
π― What it does: A domain generalization framework that maintains uncertainty in generalization strategies during the training phase (Imprecise Domain Generalisation) is proposed, along with a corresponding Imprecise Risk Optimisation (IRO) algorithm.
Domain-wise Data Acquisition to Improve Performance under Distribution Shift
Yue He (Tsinghua University), Peng Cui (Tsinghua University)
CodeDomain AdaptationSupervised Fine-TuningImage
π― What it does: This paper proposes a data collection framework based on Domain-wise Active Acquisition (DAA) to enhance the generalization performance of models in the target test domain by dynamically allocating a limited sampling budget across multiple domains in the context of distribution shift.
DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training
Zhongkai Hao (Tsinghua University), Jun Zhu (Tsinghua University)
CodeOptimizationComputational EfficiencyData-Centric LearningTransformerTime SeriesSequentialBenchmarkPhysics Related
π― What it does: Designed and implemented DPOT - a self-regressive denoising pre-trained Fourier attention transformer, used to learn universal solvers from PDE data of various dimensions, scales, and geometries.
DPZero: Private Fine-Tuning of Language Models without Backpropagation
Liang Zhang (ETH Zurich), Niao He
CodeOptimizationSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper studies and implements a zero-order fine-tuning method with differential privacy without using backpropagation, enabling efficient and private fine-tuning on large language models.
π― What it does: This paper proposes a general framework called Diffusion Reconstruction Contrastive Training (DRCT) to enhance the generalization ability of detectors for images generated by diffusion models.
π― What it does: This paper proposes a complex embedding framework for directed graphs called DUPLEX, which utilizes a dual GAT encoder to generate complex embeddings consisting of amplitude and phase components. It reconstructs the Hermitian adjacency matrix through two non-parametric decoders in a self-supervised manner, achieving high-quality representations for low-degree nodes, generalizability to new nodes, and task-independent embeddings.
Dynamic Correlation Clustering in Sublinear Update Time
Vincent Cohen-Addad (Google Research), Nikos Parotsidis (Google Research)
CodeGraphTabular
π― What it does: A relevant clustering algorithm that maintains constant approximation in dynamic node streams is proposed, with an update time of polynomial logarithmic order;
Dynamic Spectral Clustering with Provable Approximation Guarantee
Steinar Laenen (University of Edinburgh), He Sun (University of Edinburgh)
CodeGraph Neural NetworkGraph
π― What it does: This paper studies clustering algorithms for dynamic evolving graphs and proposes a dynamic spectral clustering algorithm that can effectively approximate the clustering structure in the final state of the graph.
Early Time Classification with Accumulated Accuracy Gap Control
Liran Ringel (Technion Israel Institute of Technology), Yaniv Romano (Technion Israel Institute of Technology)
CodeClassificationOptimizationTextTime Series
π― What it does: This paper proposes a statistical framework that utilizes data-driven stopping rules to achieve controlled accuracy differences in early time classification.
CodeComputational EfficiencyTransformerLarge Language ModelText
π― What it does: The EE-LLM framework is proposed, supporting early exit large language models for large-scale training and inference, and achieving 3D parallelism on Megatron-LM.
π― What it does: Utilize offline data to learn a closed-form behavior policy, enhancing the unbiased Monte Carlo evaluation variance of the target policy and reducing the required amount of online interactions.
π― What it does: An efficient precision and recall (eP&R) metric is proposed, which significantly reduces the computational cost of evaluating generative models through hubness-aware sampling.
EiG-Search: Generating Edge-Induced Subgraphs for GNN Explanation in Linear Time
Shengyao Lu (University of Alberta), Di Niu (University of Alberta)
CodeExplainability and InterpretabilityComputational EfficiencyGraph Neural NetworkGraph
π― What it does: A training-independent, linear-time subgraph explanation method called EiG-Search is proposed, which directly ranks edges by importance and searches for the best subgraph to explain GNN predictions.
Enabling Few-Shot Learning with PID Control: A Layer Adaptive Optimizer
Le Yu (Southeast University), Fir Dunkin (Southeast University)
CodeOptimizationMeta LearningImage
π― What it does: A layer adaptive PID (LA-PID) optimizer is proposed under the MAML framework for inner-loop gradient updates in few-shot learning tasks.
π― What it does: A systematic evaluation and classification of encoding methods for predictive NAS is conducted, proposing the FLAN predictor that integrates graph convolution, graph attention, and operation embedding, and designs a unified encoding to achieve transfer learning across search spaces, tasks, and datasets.
Energy-based Backdoor Defense without Task-Specific Samples and Model Retraining
Yudong Gao (China University of Petroleum), Huajie Shao (College of William and Mary)
CodeAnomaly DetectionImageTextAudio
π― What it does: This paper proposes two energy statistics-based backdoor defense methods, EBBA and EBBA+, which can achieve backdoor model detection, trigger detection, and backdoor removal without task-specific samples and model retraining.
π― What it does: Proposes the Energy-Guided Diffusion Sampling (EDIS) method, which utilizes diffusion models to generate samples guided by energy that match the online distribution, thereby alleviating the data distribution shift problem in offline-online reinforcement learning.
π― What it does: This paper proposes the DISGEN framework, which enhances data through size-invariant and task-invariant transformations on graphs, and introduces a decoupling loss to separate size information from task information in graph representations, thereby improving the generalization ability of GNNs on larger graphs.
Enhancing Sufficient Dimension Reduction via Hellinger Correlation
SeungBeom Hong, Jun Song (Korea University)
CodeTabular
π― What it does: A sufficient dimension reduction (SDR) method based on Hellinger correlation is proposed and applied to single-index models; this method can serve as an enhancement to existing SDR algorithms (SIR, SAVE, DR, MAVE, etc.).
Ensemble Pruning for Out-of-distribution Generalization
Fengchun Qiao (University of Delaware), Xi Peng (University of Delaware)
CodeDomain AdaptationOptimizationImage
π― What it does: A model predictive relationship topology-based integrated pruning framework is proposed, which enhances out-of-distribution generalization performance by selecting diverse subsets in the absence of target labels.
π― What it does: Proposes the EquiAV framework, which combines equivariance and self-supervised contrastive learning for joint representation learning of audio and visual modalities.
π― What it does: A Minimal Frame Averaging (MFA) framework is proposed, which achieves precise equivariance through the construction of a minimal frame with a single forward inference.
Equivariant Frames and the Impossibility of Continuous Canonicalization
Nadav Dym (Technion), Jonathan W. Siegel
CodeClassificationPoint Cloud
π― What it does: This study investigates the relationship between frame averaging and equivariant networks, proving that commonly used unweighted frames often lead to continuity destruction. It proposes weighted and weakly equivariant robust frames, constructing robust frames that can maintain continuity under three common group actions (S_n, SO_d, O_d).
Equivariant Graph Neural Operator for Modeling 3D Dynamics
Minkai Xu (Stanford University), Anima Anandkumar (California Institute of Technology)
CodeGraph Neural NetworkGraphTime SeriesPhysics Related
π― What it does: A new Equivariant Graph Neural Operator (EGNO) is proposed, which simulates three-dimensional dynamics by learning SE(3) equivariant time trajectory mappings, replacing traditional GNN methods that only predict the next state.
π― What it does: This paper proposes a method that utilizes an Error Feedback mechanism to compress the gradient history window of full matrix preprocessors (such as M-FAC and GGT), significantly reducing memory consumption without affecting convergence.
CodeDrug DiscoveryProtein Structure PredictionTransformerLarge Language ModelSupervised Fine-TuningBiomedical Data
π― What it does: A multi-scale protein language model ESM-AA is proposed, capable of unified modeling at both residue and atomic scales, suitable for tasks involving protein-small molecule interactions.
Jan Pauls (University of MΓΌnster), Fabian Gieseke (University of Copenhagen)
CodeSegmentationGenerationOptimizationConvolutional Neural NetworkImageAgriculture Related
π― What it does: Based on Sentinel-1/2 satellite imagery and GEDI LiDAR measurements, a fully convolutional network was trained using U-Net+ResNet50 to generate global canopy height maps with a resolution of 10 meters.
π― What it does: A parameter-efficient fine-tuning method based on hyperplane reflection (Householder transformation) called ETHER and its relaxed version ETHER+ is proposed, which can efficiently adapt large pre-trained models without altering the model's pre-trained knowledge.
Evaluation of Trajectory Distribution Predictions with Energy Score
Novin Shahroudi (University of Tartu), Meelis Kull (University of Tartu)
CodeTime SeriesSequential
π― What it does: This paper conducts a theoretical analysis and experimental validation of evaluation metrics for trajectory distribution prediction, proposing a rigorous and effective evaluation method based on the Energy Score.
EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs
Haohui Wang (Virginia Tech), Dawei Zhou (Virginia Tech)
CodeDomain AdaptationRepresentation LearningGraph Neural NetworkTransformerGraphTime Series
π― What it does: This paper studies the problem of transfer learning on dynamic non-IID graphs and proposes the EVOLUNET framework to achieve knowledge transfer from source graphs to target graphs.
Exact Soft Analytical Side-Channel Attacks using Tractable Circuits
Thomas Wedenig (Graz University of Technology), Robert Peharz (Graz University of Technology)
CodeOptimizationSafty and PrivacyTime Series
π― What it does: This paper proposes an exact side-channel attack algorithm based on compileable circuits (SDD/PSDD) called ExSASCA, which replaces traditional loop belief propagation to achieve precise inference of the first round of AES-128 encryption.
ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking
Wenshuo Li (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab)
CodeCompressionTransformerLarge Language ModelImageText
π― What it does: Proposes the ExCP framework, which performs extreme compression of checkpoints during the training process of LLMs, achieving nearly lossless compression using residuals, weight-momentum joint pruning, and non-uniform quantization.
Exploration and Anti-Exploration with Distributional Random Network Distillation
Kai Yang (Tsinghua University), Xiu Li (Tsinghua University)
CodeReinforcement Learning
π― What it does: This paper studies and addresses the issue of inconsistent rewards in Random Network Distillation (RND), proposing Distributional Random Network Distillation (DRND). It distills multiple random target networks and utilizes pseudo-counts to achieve more accurate exploration rewards, applying this method to online PPO and offline SAC in reinforcement learning.
Exploring the Low-Pass Filtering Behavior in Image Super-Resolution
Haoyu Deng (University of Electronic Science and Technology of China), Liang-Jian Deng (University of Electronic Science and Technology of China)
CodeRestorationSuper ResolutionImage
π― What it does: This paper studies the low-pass filtering behavior of single-image super-resolution networks through signal processing theory, discovering that the impulse response approximates a sinc function, and proposes the HyRA analysis framework and the FSDS spectral similarity evaluation metric.
Extending Test-Time Augmentation with Metamorphic Relations for Combinatorial Problems
Siwei Wei (Chinese Academy of Sciences), Yan Cai (Chinese Academy of Sciences)
CodeOptimizationGraph Neural NetworkGraphTabular
π― What it does: This paper proposes the MAGG method, which enhances incremental reasoning of pre-trained machine learning models during the inference phase by utilizing the metamorphic relations of combinatorial problems;
π― What it does: An asynchronous adaptive federated learning algorithm FADAS is proposed to address the slow client bottleneck caused by traditional synchronous aggregation, incorporating a delay-adaptive learning rate based on a maximum delay threshold.
FAFE: Immune Complex Modeling with Geodesic Distance Loss on Noisy Group Frames
Ruidong Wu (Helixon), Jian Peng (Tsinghua)
CodeProtein Structure PredictionSupervised Fine-TuningBiomedical Data
π― What it does: A new geometric distance loss F2E is proposed to address the gradient vanishing problem caused by the FAPE loss in AlphaFold2-Multimer during immune complex modeling, further improved to a group-level F2E (G-F2E) to simultaneously consider rotational and translational errors.
π― What it does: A general multi-dimensional multi-group fair risk control framework (s, G, Ξ±)-GMC is proposed, along with an algorithm based on projection updates.
π― What it does: A time-conditioned latent audio diffusion model (Stable Audio) is proposed, capable of generating up to 95 seconds of 44.1kHz stereo music and sound effects from text and duration prompts, supporting variable length control.
Fast, Scalable, Warm-Start Semidefinite Programming with Spectral Bundling and Sketching
Rico Angell (University of Massachusetts Amherst), Andrew McCallum (University of Massachusetts Amherst)
CodeOptimizationTabular
π― What it does: A unified spectral bundling and matrix sampling method, USBS, has been designed and implemented to solve large-scale semidefinite programming (SDP) problems with equality and inequality constraints, supporting warm-start initialization.
π― What it does: Proposes two low-rank model training frameworks, FedLMT and pFedLMT, to address the issue of system heterogeneity in federated learning;
CodeClassificationContrastive LearningImageBiomedical Data
π― What it does: A FALCON method is proposed, which automatically discovers fine-grained categories and learns fine-grained classifiers under unsupervised conditions using only coarse-grained labels.
π― What it does: A flexible visual Transformer (FiT) capable of image generation at arbitrary resolutions and aspect ratios is proposed and applied to diffusion models;
FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Prediction
Zhonghang Li (South China University of Technology), Chao Huang (University of Hong Kong)
CodeKnowledge DistillationGraph Neural NetworkPrompt EngineeringTime Series
π― What it does: Proposes FlashST, a lightweight spatiotemporal Prompt-Tuning framework for quickly adapting pre-trained models to various urban traffic prediction tasks;
π― What it does: The Floating Anchor Diffusion (FADiff) model is proposed for the design of multifunctional motif protein scaffolds, allowing motifs to maintain rigidity and float freely during the diffusion process, automatically determining their positions within the protein.