🎯 What it does: A time-based spiking neural network (GRSNN) based on synaptic delay is proposed for reasoning tasks in knowledge graphs and general graphs.
TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors
Yichuan Mo (Peking University), Yisen Wang (Peking University)
CodeGenerationData SynthesisSafty and PrivacyDiffusion modelImageStochastic Differential Equation
🎯 What it does: A unified framework named TERD is proposed for identifying and eliminating backdoor attacks in diffusion models, including the reverse recovery of triggers and detection of inputs/models.
Test-Time Degradation Adaptation for Open-Set Image Restoration
Yuanbiao Gou (Sichuan University), Xi Peng (Sichuan University)
CodeRestorationDiffusion modelImage
🎯 What it does: A test-time adaptive open-set image restoration framework TAO is proposed, which combines a pre-trained diffusion model, a test-time degradation adapter, and a dynamic guidance strategy to achieve image restoration for single-sample unknown degradations.
🎯 What it does: This paper proposes a testing-time adaptation method called FOA, which utilizes only forward propagation, employing input prompt learning and activation shifting to achieve online adaptation to distribution drift.
The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents
Yatin Dandi (Ecole Polytechnique Federale de Lausanne), Florent Krzakala (Ecole Polytechnique Federale de Lausanne)
CodeOptimizationTabular
🎯 What it does: This paper studies the use of multi-pass gradient descent (GD) with repeated batches of the same data in a two-layer neural network, exploring its dynamic performance when learning multi-objective functions.
The Merit of River Network Topology for Neural Flood Forecasting
Nikolas Kirschstein (University of Oxford), Yixuan Sun (Technical University of Munich)
CodeGraph Neural NetworkGraphTime Series
🎯 What it does: This paper studies the impact of river network topology information on river flow prediction based on graph neural networks (GNNs). It constructs a GNN model based on the LamaH-CE dataset and compares the effects of different adjacency definitions (no adjacency, binary adjacency, physical weights, learned weights) on prediction performance.
The Privacy Power of Correlated Noise in Decentralized Learning
Youssef Allouah (École Polytechnique Fédérale de Lausanne), Rachid Guerraoui (École Polytechnique Fédérale de Lausanne)
CodeFederated LearningSafty and PrivacyImageTabular
🎯 What it does: The DECOR algorithm is proposed, which achieves differential privacy in decentralized learning by introducing mutually canceling correlated Gaussian noise.
TIC-TAC: A Framework For Improved Covariance Estimation In Deep Heteroscedastic Regression
Megh Shukla (Ecole Polytechnique Federale de Lausanne), Alexandre Alahi (Ecole Polytechnique Federale de Lausanne)
CodePose EstimationOptimizationTabular
🎯 What it does: This paper proposes two methods that utilize the gradient and curvature from Taylor expansion to approximate the covariance of predictive distributions (Taylor Induced Covariance, TIC), and introduces the Task Agnostic Correlations (TAC) metric to evaluate the quality of covariance, thereby achieving better covariance estimation and training convergence in deep heteroscedastic regression.
Timer: Generative Pre-trained Transformers Are Large Time Series Models
Yong Liu (Tsinghua University), Mingsheng Long (Tsinghua University)
CodeAnomaly DetectionTransformerLarge Language ModelSupervised Fine-TuningTime Series
🎯 What it does: A unified time series dataset UTSD containing 1 billion temporal points was constructed, proposing a single series sequence (S3) format to unify heterogeneous multivariate time series. A GPT-style autoregressive pre-trained Transformer (Timer) was used and fine-tuned and evaluated on multiple tasks such as prediction, missing imputation, and anomaly detection.
🎯 What it does: This paper proposes TimeSiam, a self-supervised pre-training framework based on Siamese networks, which reconstructs the current subsequence using past subsequences to learn temporal correlations.
🎯 What it does: TinyTrain implements adaptive sparse training with a small amount of labeled data on resource-constrained edge devices, significantly reducing memory and computational overhead while maintaining high accuracy.
To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO
Zi-Hao Qiu (Nanjing University), Tianbao Yang (Texas A&M University)
CodeRetrievalOptimizationTransformerLarge Language ModelContrastive LearningImageText
🎯 What it does: This paper proposes a temperature network framework (TempNet) based on Distributed Robust Optimization (DRO), which can automatically predict personalized temperatures for each sample during the training and inference of large models (LLM and CLIP), enhancing model performance.
Yongcheng Zeng (Institute of Automation, Chinese Academy of Sciences), Jun Wang (University College London)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A Token-level Direct Preference Optimization (TDPO) method for aligning LLMs is proposed, utilizing forward KL constraints to control the bias of each token and achieving direct policy optimization through tokenization of the Bradley-Terry model.
Yifan Zhu (Academy of Mathematics and Systems Science, Chinese Academy of Sciences), Xiao-Shan Gao (Academy of Mathematics and Systems Science, Chinese Academy of Sciences)
CodeOptimizationAdversarial AttackPoint Cloud
🎯 What it does: A poisoning method based on Feature Collision Error Minimization (FC-EM) is proposed for the availability attack on 3D point clouds.
Towards a Self-contained Data-driven Global Weather Forecasting Framework
Yi Xiao (Tsinghua University), Wanli Ouyang (Shanghai Artificial Intelligence Laboratory)
CodeOptimizationAuto EncoderTime Series
🎯 What it does: An AI-embedded 4D variational data assimilation algorithm (AI-embedded 4DVar) is proposed and implemented, which is coupled with the global AI weather forecasting model FengWu to construct a self-consistent full-scale weather forecasting framework.
Towards Efficient Training and Evaluation of Robust Models against $l_0$ Bounded Adversarial Perturbations
Xuyang Zhong (City University of Hong Kong), Chen Liu (City University of Hong Kong)
CodeOptimizationAdversarial AttackImage
🎯 What it does: This paper proposes a white-box attack method based on sparse-PGD, which can efficiently generate l0-constrained sparse adversarial perturbations and combines with black-box attacks to form sAA for comprehensive evaluation; at the same time, this attack is used for adversarial training, resulting in a model with the strongest robustness against sparse attacks.
Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error
Haoran Li (University of Chinese Academy of Sciences), Shichen Liao (University of Chinese Academy of Sciences)
CodeOptimizationReinforcement LearningVideo
🎯 What it does: This study investigates the existence of optimal robust policies in the State Adversarial Markov Decision Process (SA-MDP) and proves that the Bellman optimal policy is indeed the optimal robust policy. It then proposes the Consistent Adversarial Robust DQN (CAR-DQN) algorithm, which utilizes the Bellman ∞-error.
🎯 What it does: This paper proposes a maximum likelihood dataset reconstruction attack based on constraint programming, which can reconstruct the training set using only the structure and node count information of the random forest.
Training Greedy Policy for Proposal Batch Selection in Expensive Multi-Objective Combinatorial Optimization
Deokjae Lee (Seoul National University), Kyunghyun Cho (New York University)
CodeOptimizationReinforcement LearningSequential
🎯 What it does: In the expensive multi-objective combinatorial optimization (MOCO) problem, a greedy strategy was trained to directly perform subset selection on a batch of candidate sets within the combinatorial space, thereby optimizing the batch acquisition function.
Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
Zhiheng Xi (Fudan University), Xuanjing Huang (Fudan University)
CodeLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes a training method R³ based on inverse curriculum reinforcement learning, which achieves stepwise supervision in reasoning tasks using only the final outcome for supervision.
Training-Free Long-Context Scaling of Large Language Models
Chenxin An (University of Hong Kong), Lingpeng Kong
CodeTransformerLarge Language ModelText
🎯 What it does: This paper proposes a Dual Chunk Attention (DCA) mechanism that does not require further training, achieving efficient inference of LLaMA2 70B with context lengths exceeding 100k.
Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models
Akhil Kedia (Samsung Research), Haejun Lee (Samsung Research)
CodeTransformerLarge Language ModelImageTextAudio
🎯 What it does: This paper proposes a unified signal propagation theory and provides closed-form expressions for the first and second moments during the forward and backward propagation of each layer in the Transformer, aimed at explaining and alleviating issues such as gradient vanishing/explosion, rank collapse, and instability of attention scores in deep Transformers.
Transolver: A Fast Transformer Solver for PDEs on General Geometries
Haixu Wu (Tsinghua University), Mingsheng Long (Tsinghua University)
CodeTransformerMeshBenchmarkPhysics Related
🎯 What it does: This paper presents Transolver, a Transformer structure utilizing Physics-Attention, capable of quickly solving partial differential equations on complex geometric meshes.
Sina Akbari (École Polytechnique Fédérale de Lausanne), Negar Kiyavash (École Polytechnique Fédérale de Lausanne)
CodeTabularElectronic Health Records
🎯 What it does: This paper proposes and derives a Triple Changes estimator for identifying potential outcome distributions and their average treatment effects in policy evaluation targeted at subpopulations.
🎯 What it does: A Triplet Graph Transformer (TGT) is designed, which directly implements third-order information flow between triplet nodes through a triplet interaction mechanism, and constructs a three-stage training process for distance predictor and attribute predictor, supporting random inference and uncertainty estimation.
🎯 What it does: The Tripod model is proposed, which integrates three complementary inductive biases to achieve unsupervised decomposable representation learning.
TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Tasks
Zhiruo Wang, Daniel Fried (Carnegie Mellon University)
CodeOptimizationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTabular
🎯 What it does: Dynamically generate, maintain, and prune a reusable high-level function toolbox through LLM under unsupervised and untrained conditions, in order to produce more concise and accurate programmatic answers.
🎯 What it does: The ZKAUDIT protocol is proposed, which enables trustworthy auditing of deep learning models and training processes without disclosing model weights or training data.
Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation
Zhenyu He (Peking University), Di He (Peking University)
CodeTransformerText
🎯 What it does: This paper proposes Bilevel Positional Encoding (BiPE), a dual-layer position encoding that combines intra-segment encoding and inter-segment encoding to enhance the length extrapolation capability of Transformers on long sequences.
UGrid: An Efficient-And-Rigorous Neural Multigrid Solver for Linear PDEs
Xi Han (Stony Brook University), Hong Qin (Stony Brook University)
CodeConvolutional Neural NetworkMeshPhysics Related
🎯 What it does: A neural multigrid solver UGrid based on U-Net and multigrid fusion is proposed and implemented, providing rigorous convergence and correctness proofs for linear PDEs.
🎯 What it does: A unified label refinement framework ULAREF is proposed, which predicts reliability through global detection and performs local enhancement on unreliable samples, achieving a unified treatment of noisy labels and partial labels.
🎯 What it does: The Density Aware Evidential Deep Learning (DAEDL) method is proposed, which improves the performance of traditional Evidential Deep Learning (EDL) in terms of OOD detection and classification accuracy.
Understanding the Learning Dynamics of Alignment with Human Feedback
Shawn Im (University of Wisconsin Madison), Yixuan Li (University of Wisconsin Madison)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: The theoretical analysis of the learning dynamics of Direct Preference Optimization (DPO) on large language models is conducted, with experimental validation on models such as Llama-2 and Mistral.
🎯 What it does: The PromptGIP model is proposed, unifying low-level image processing (image restoration, enhancement, edge detection, etc.) into a visual prompt question-answering framework, enabling a single model to support multiple tasks.
🎯 What it does: This paper proposes a concept balancing technique called CoBalT based on unsupervised object-centric learning, aimed at alleviating short-sighted correlations in training data without the need for manual subgroup labels.
🎯 What it does: This paper proposes an unsupervised domain adaptation method for ultrasound fetal structure detection, ToMo-UDA, which combines topological and morphological knowledge transfer.
Using AI Uncertainty Quantification to Improve Human Decision-Making
Laura Marusich, Murat Kantarcioglu (University of Texas at Dallas)
CodeClassificationTabular
🎯 What it does: This study evaluates the impact of providing human decision-makers with high-quality, calibrated instance-level AI uncertainty quantification (UQ) information on decision accuracy and confidence calibration.
Pengyi Li (Tianjin University), Fazl Barez (University of Oxford)
CodeReinforcement Learning
🎯 What it does: The VEB-RL framework is proposed, which combines evolutionary algorithms with value-based reinforcement learning by maintaining a Q-network within the population, using negative TD error to evaluate individuals, and employing elite interaction to enhance sample quality, thereby improving value function learning and policy performance.
Variational Learning is Effective for Large Deep Networks
Yuesong Shen (Technical University of Munich), Thomas Möllenhoff (RIKEN Center for AI Project)
CodeOptimizationTransformerLarge Language ModelImageText
🎯 What it does: An improved variational online Newton optimizer IVON is proposed and implemented, capable of directly optimizing variational objectives on large-scale deep networks (such as GPT-2, ResNet), while enhancing accuracy and uncertainty estimation with a computational cost similar to Adam.
🎯 What it does: This paper proposes a Variational Schrödinger Diffusion Model (VSDM), which linearizes the forward score through variational inference to eliminate the simulation overhead of backward training, and constructs a scalable multivariate diffusion generative framework by utilizing stochastic approximation for adaptive optimization of the forward score.
🎯 What it does: A weakly convex regularization framework is proposed, and its convergence at critical points is proven. Additionally, the convergence and asymptotic rate of the forward-backward gradient algorithm based on weakly convex regularization are provided, and this framework is implemented as an Input Weakly Convex Neural Network (IWCNN) for CT reconstruction.
Weakly-Supervised Residual Evidential Learning for Multi-Instance Uncertainty Estimation
Pei Liu (University of Electronic Science and Technology of China), Luping Ji (University of Electronic Science and Technology of China)
CodeClassificationAnomaly DetectionConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: This paper proposes a weakly supervised uncertainty estimation framework for multi-instance learning (MIL) called MIREL, which models uncertainty at both the bag level and instance level using residual evidence learning under the condition of not having complete instance labels.
Yang Zhao (Tsinghua University), Xiuyuan Hu (Tsinghua University)
CodeOptimizationTransformerImage
🎯 What it does: This paper investigates the reasons for performance degradation when gradient regularization (GR) is used in conjunction with adaptive optimizers (such as Adam and RMSProp) and learning rate warmup (LR warmup), and proposes three GR warmup strategies to mitigate this issue.
Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Models
Xavier Suau (Apple), Pau Rodriguez
CodeGenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a hyperparameter-free AURA intervention method to reduce toxic generation by identifying and suppressing expert neurons in language models responsible for producing toxic language, while maintaining overall model performance.
Winner-takes-all learners are geometry-aware conditional density estimators
Victor Letzelter (Valeo), Patrick Perez
CodeMixture of ExpertsTabularAudio
🎯 What it does: A conditional density estimation method based on Winner-takes-all (WTA) training is proposed—Voronoi-WTA, which constructs adaptive Voronoi partitions using the multi-head outputs predicted by WTA, and employs truncated kernel density estimation within each partition;
WISER: Weak Supervision and Supervised Representation Learning to Improve Drug Response Prediction in Cancer
Kumar Shubham (Indian Institute of Science), Vaibhav Rajan (National University of Singapore)
CodeRepresentation LearningDrug DiscoveryGenerative Adversarial NetworkContrastive LearningBiomedical Data
🎯 What it does: Developed the WISER framework, which combines weak supervision with supervised domain-invariant representation learning to improve drug response prediction in cancer patients.
🎯 What it does: By using nested tokenization and a two-stage pipeline, the existing visual models that only process small images can integrate large-scale context of the entire image while maintaining local details, achieving classification, detection, and segmentation of large images.
Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement
Che Liu (Imperial College London), Rossella Arcucci (Imperial College London)
CodeClassificationRepresentation LearningConvolutional Neural NetworkLarge Language ModelContrastive LearningMultimodalityBiomedical DataElectrocardiogram
🎯 What it does: We propose a multi-modal learning-based ECG representation learning framework MERL, which can achieve zero-shot ECG classification by utilizing cross-modal alignment between ECG signals and corresponding reports.
Zero-Shot Reinforcement Learning via Function Encoders
Tyler Ingebrand (University of Texas at Austin), ufuk topcu
CodeReinforcement Learning
🎯 What it does: This paper proposes a Function Encoder to represent reward or transition functions as a weighted combination of several learned nonlinear basis functions, resulting in a unique vector representation, which is then used as contextual input for any reinforcement learning algorithm to achieve zero-shot transfer.