π― What it does: This paper proposes the Token Statistics Transformer (TOST), which uses a white-box design based on the MCR-2 variational objective to construct a Token Statistics Self-Attention (TSSA) module that does not require pairwise similarity calculations and has a linear growth in computational and memory complexity, replacing traditional self-attention to achieve an efficient Transformer.
TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
Haiyang Wang (Peking University), Bernt Schiele (Max Planck Institute for Informatics)
CodeTransformerLarge Language ModelImageText
π― What it does: Tokenformer is proposed, utilizing attention to treat model parameters as scalable tokens, replacing traditional linear projections, and supporting gradual model expansion without the need for retraining from scratch.
TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models
Ziyao Shangguan (Yale University), Arman Cohan (Allen Institute for AI)
CodeTransformerLarge Language ModelVideoMultimodalityBenchmark
π― What it does: A new video understanding benchmark called TOMATO is proposed and constructed, specifically to evaluate the visual temporal reasoning capabilities of multimodal foundational models.
Tool-Planner: Task Planning with Clusters across Multiple Tools
Yanming Liu (Zhejiang University), Tianyu Du (Zhejiang University)
CodeTransformerLarge Language ModelPrompt EngineeringBenchmark
π― What it does: A task planning framework called Tool-Planner based on tool clustering is proposed for more efficient invocation and error correction of external APIs in large language models.
ToolDial: Multi-turn Dialogue Generation Method for Tool-Augmented Language Models
Jeonghoon Shim (Seoul National University), Yohan Jo (Seoul National University)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Constructed and released the ToolDial dataset, which contains 11,111 multi-turn dialogues for evaluating Tool-Augmented Language Models (TALM) in real-world scenarios regarding dialogue state tracking, action prediction, and answer credibility.
ToolGen: Unified Tool Retrieval and Calling via Generation
Renxi Wang (LibrAI), Haonan Li (Mohamed bin Zayed University of Artificial Intelligence)
CodeGenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Directly embed tool knowledge into the LLM vocabulary, using virtual tokens to achieve a unified generation process for tool retrieval and invocation;
Topograph: An Efficient Graph-Based Framework for Strictly Topology Preserving Image Segmentation
Laurin Lux (Technical University of Munich), Johannes C. Paetzold (Weill Cornell Medicine)
CodeSegmentationConvolutional Neural NetworkImage
π― What it does: A component graph-based graphical framework is proposed to enforce topological consistency during the training of segmentation networks, which can efficiently identify and correct topological errors.
TopoLM: brain-like spatio-functional organization in a topographic language model
Neil Rathi (EPFL), Martin Schrimpf (EPFL)
CodeTransformerLarge Language ModelTextBiomedical DataMagnetic Resonance Imaging
π― What it does: We propose and train TopoLM, a language model that incorporates two-dimensional spatial encoding and spatial smoothness loss into the Transformer, achieving the spatial functional organization of the brain's language system.
Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity
Yam Eitan (Technion Israel Institute of Technology), Haggai Maron (Technion Israel Institute of Technology)
CodeGraph Neural NetworkGraphBenchmark
π― What it does: This paper first analyzes the expressive power of the Higher-Order Message Passing (HOMP) framework from a topological perspective, proving that it cannot distinguish fundamental topological/metric invariants such as diameter, orientability, planarity, and homology groups. Subsequently, it proposes Multi-Cell Networks (MCN) and its scalable version, Scalable Multi-Cell Networks (SMCN), which achieve full expressiveness by introducing multi-cell covariant layers, significantly enhancing the ability to discern the aforementioned invariants while maintaining computational efficiency. Based on this, three novel CC benchmarks (Torus dataset, cross diameter and second Betti number prediction tasks of Lifted ZINC) are constructed, and comparisons are made with traditional GNNs, HOMP, and other high-expressiveness models on these benchmarks and real graph datasets (ZINC-12K, MOLHIV, MOLESOL). SMCN outperforms the control group in metrics such as MAE, ROC-AUC, and RMSE, achieving a cross diameter prediction accuracy of 92.8% and a second Betti number prediction accuracy of 99.6%.
Topological Zigzag Spaghetti for Diffusion-based Generation and Prediction on Graphs
Yuzhou Chen (University of California), Yulia Gel
CodeGenerationDrug DiscoveryGraph Neural NetworkDiffusion modelGraphTime SeriesBiomedical Data
π― What it does: This paper proposes a new method for graph diffusion generation and prediction, centered around a 'zigzag spaghetti (ZS)' topological summary constructed based on zigzag persistence, which captures high-order topological features of multi-scale temporal evolution graphs and embeds them into the graph diffusion model.
π― What it does: The TORCHTITAN framework is proposed, which unifies and extends the native distributed training technology of PyTorch, creating a one-stop solution for LLM pre-training.
π― What it does: This paper constructs a large dataset containing 177 subjects and a total of 3,127 image-fMRI pairs, and proposes a unified whole-brain decoding framework capable of retrieving visual information from unseen subjects.
π― What it does: Condition Contrastive Alignment (CCA) is proposed, which significantly improves image generation quality without guidance (no CFG) through a fine-tuning process on a pre-trained autoregressive visual generative model, while halving the sampling cost.
Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders
Qichao Shentu (East China Normal University), Chenjuan Guo (East China Normal University)
CodeAnomaly DetectionTransformerAuto EncoderGenerative Adversarial NetworkTime Series
π― What it does: A general-purpose time series anomaly detection model DADA has been developed, which can be directly applied to multi-domain target datasets in zero-shot scenarios.
Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
Zeyu Gan (Renmin University of China), Yong Liu (Renmin University of China)
CodeGenerationData SynthesisLarge Language ModelText
π― What it does: This study investigates the role of synthetic data in post-training of LLMs, establishes a distribution model for synthetic data generation, proposes an inverse bottleneck framework, and derives an upper bound on generalization error using information theory.
Towards Auto-Regressive Next-Token Prediction: In-context Learning Emerges from Generalization
Zixuan Gong (Renmin University of China), Yong Liu (Renmin University of China)
CodeGenerationOptimizationTransformerLarge Language ModelTextStochastic Differential Equation
π― What it does: A full-process pre-training and ICL framework based on the autoregressive next word prediction (AR-NTP) paradigm is proposed, and a two-layer expected PAC-Bayesian generalization upper bound is provided, proving that the ICL capability arises from the generalization of sequences and topics.
π― What it does: A dual-head deep clustering network (CDC) is proposed, which simultaneously calibrates the confidence of the model output during the unsupervised clustering process and dynamically selects high-confidence samples for pseudo-label self-training using the calibrated confidence, significantly improving clustering accuracy and the reliability of confidence.
Towards Continuous Reuse of Graph Models via Holistic Memory Diversification
Ziyue Qiao (Great Bay University), Hui Xiong (Hong Kong University of Science and Technology)
CodeGraph Neural NetworkGraph
π― What it does: This paper proposes an incremental learning method DMSG for the ever-growing graph structured data, which can continuously train the model when new tasks arise while retaining knowledge from old tasks.
Towards Effective Evaluations and Comparisons for LLM Unlearning Methods
Qizhou Wang (Hong Kong Baptist University), Masashi Sugiyama (RIKEN Center for Advanced Intelligence Project)
CodeLarge Language ModelText
π― What it does: This paper proposes a unified evaluation framework UWC to measure the 'unlearning' effect of large language models and to fairly compare existing unlearning methods.
Towards Explaining the Power of Constant-depth Graph Neural Networks for Structured Linear Programming
Qian Li (Shenzhen International Center for Industrial and Applied Mathematics), Ruoyu Sun (School of Data Science, The Chinese University of Hong Kong)
CodeOptimizationGraph Neural NetworkGraph
π― What it does: This paper proves that graph neural networks (GNNs) with constant depth and constant width can effectively solve sparse binary linear programming problems, and provides an implementation and experimental validation based on distributed algorithms.
Towards Fast, Specialized Machine Learning Force Fields: Distilling Foundation Models via Energy Hessians
Ishan Amin (University of California), Aditi S. Krishnapriyan (University of California)
CodeComputational EfficiencyKnowledge DistillationGraph Neural NetworkBiomedical Data
π― What it does: The study proposes a method for transferring representations from large-scale foundational models (FM) to smaller, faster specialized machine learning force fields (MLFF) through knowledge distillation of the energy Hessian matrix, significantly accelerating inference while maintaining or improving accuracy.
Towards Federated RLHF with Aggregated Client Preference for LLMs
Feijie Wu (Purdue University), Jing Gao (State University of New York at Albany)
CodeFederated LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextBenchmark
π― What it does: This study proposes the use of federated learning to collect diverse user preferences and implements RLHF alignment for large language models through a binary selector, constructing two frameworks: FedBis and FedBiscuit.
Towards Foundation Models for Mixed Integer Linear Programming
Sirui Li (Massachusetts Institute of Technology), Beibin Li (Microsoft Research)
CodeOptimizationGraph Neural NetworkLarge Language ModelContrastive LearningTextChain-of-Thought
π― What it does: A LLM-based MILP-Evolve evolutionary framework is proposed to generate a vast and diverse range of MILP classes, and on this basis, a single GNN+attention-based model is trained, which can achieve multi-class generalization in three major tasks: predicting the integrality gap of mixed-integer linear programming, branch learning, and MILP-text alignment.
Towards General-Purpose Model-Free Reinforcement Learning
Scott Fujimoto (Meta Platforms), Michael Rabbat (Meta Platforms)
CodeReinforcement LearningTabularBenchmark
π― What it does: A general model-free reinforcement learning algorithm MR.Q is designed, which utilizes model-based representation learning to generate approximately linear value function state-action embeddings, and is applicable to various observation and action spaces under a single hyperparameter setting.
π― What it does: This paper proposes an interpretable causal self-adaptive representation method (CSR) for rapid transfer and adaptation of reinforcement learning models in the scenarios of distribution changes and environmental space expansion.
Towards hyperparameter-free optimization with differential privacy
Ruixuan Liu (Emory University), Zhiqi Bu (Amazon)
CodeOptimizationSafty and PrivacyTransformerLarge Language ModelImageText
π― What it does: A hyperparameter-free differential privacy optimization framework called HyFreeDP is proposed, which automatically adjusts the learning rate and uses automatic gradient clipping based on privacy loss estimation.
CodeObject DetectionExplainability and InterpretabilityTransformerVision Language ModelImage
π― What it does: This paper studies the processing of visual input in visual language models (VLMs) and finds that object information is highly localized to the corresponding visual tokens, which are gradually mapped to interpretable text embeddings at different layers.
Towards Marginal Fairness Sliced Wasserstein Barycenter
Khai Nguyen (University of Texas at Austin), Nhat Ho (University of Texas at Austin)
CodeOptimizationImagePoint Cloud
π― What it does: Proposed and solved the 'marginally fair sliced Wasserstein barycenter' problem, constraining the distances between each margin to be similar.
π― What it does: This paper proposes a cross-modal generalization framework COX that does not require instance-level modal correspondence, aiming to infer the labels of unknown modalities using the knowledge of known modalities.
π― What it does: This study investigates the robustness of Direct Preference Optimization (DPO) under point-to-point and pairwise noise, and proposes the Distributionally Robustifying DPO (Dr.DPO) framework, which enhances resistance to pairwise noise while maintaining point-to-point robustness through an additional hyperparameter Ξ²β².
Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs
Sungmin Cha (New York University), Moontae Lee (LG AI Research)
CodeTransformerLarge Language ModelText
π― What it does: This paper studies the problem of knowledge unlearning in large language models and proposes a low-rank knowledge unlearning framework called LoKU, which addresses issues of instability and excessive forgetting associated with traditional gradient ascent (GA) methods.
π― What it does: A framework for adaptive multi-modal open set testing (MM-OSTTA) called AEO is proposed and validated, which aims to enhance the entropy difference between known and unknown samples online, thereby improving unknown class detection and overall adaptation performance.
Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning
Somnath Basu Roy Chowdhury (University of North Carolina Chapel Hill), Snigdha Chaturvedi (Google Research)
CodeTransformerLarge Language ModelSupervised Fine-TuningImageText
π― What it does: This paper proposes an Exact Machine Unlearning framework called S3T, based on sharding, slicing, and Parameter-Efficient Fine-Tuning (PEFT), which enables a quick and training-free response to data deletion requests.
π― What it does: A scalable topological regularization method based on Principal Persistence Measures (PPM) is proposed and applied to tasks such as Generative Adversarial Networks (GAN) and Semi-Supervised Learning (SSL) to improve the model's generation quality and classification performance.
Towards Synergistic Path-based Explanations for Knowledge Graph Completion: Exploration and Evaluation
Tengfei Ma (Hunan University), xiangxiang Zeng
CodeExplainability and InterpretabilityKnowledge DistillationGraph Neural NetworkGraph
π― What it does: KGExplainer provides interpretable path explanations for knowledge graph completion models, identifying collaborative paths and assessing their credibility.
Towards Unbiased Learning in Semi-Supervised Semantic Segmentation
Rui Sun (University of Science and Technology of China), Tianzhu Zhang (University of Science and Technology of China)
CodeSegmentationDiffusion modelImage
π― What it does: Proposes DiffMatch, modeling the semi-supervised semantic segmentation task as a conditional discrete data generation problem, and generates pseudo-labels through a diffusion model;
TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees
Weibin Liao (Peking University), Yasha Wang (Peking University)
CodeRecommendation SystemOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: This paper proposes TPO, a method for directly learning multi-branch and multi-step preferences based on tree-shaped preference trees, as a replacement for traditional DPO.
TRACE: Temporal Grounding Video LLM via Causal Event Modeling
Yongxin Guo (Chinese University of Hong Kong), Xiaoying Tang (Chinese University of Hong Kong)
CodeGenerationRetrievalTransformerLarge Language ModelVision Language ModelVideoText
π― What it does: A causal event modeling framework is proposed, constructing the TRACE video LLM, which predicts the next event in an autoregressive manner based on preceding events, video, and text instructions, achieving zero-shot video temporal localization tasks.
Tracking objects that change in appearance with phase synchrony
Sabine Muzellec (CerCo CNRS Universite de Toulouse), Thomas Serre (Brown University)
CodeObject TrackingRecurrent Neural NetworkVideo
π― What it does: A neural synchronous attention mechanism based on complex-valued recurrent neural networks (CV-RNN) has been developed to address tracking tasks when the appearance of objects changes over time.
Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
Jun Zhang (Zhejiang University), Kunlong Zhou (Guangdong OPPO Mobile Telecommunications Corporation)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes LORAM, a memory-efficient fine-tuning framework that trains LoRA on a compressed model and infers on the full model.
Training Free Exponential Context Extension via Cascading KV Cache
Jeffrey Willette (KAIST AI), Sung Ju Hwang (KAIST AI)
CodeTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: A training-agnostic cascading KV cache is proposed, which dynamically retains important tokens and extends context length with linear complexity.
Training Large Language Models for Retrieval-Augmented Question Answering through Backtracking Correction
Huawen Feng (South China University of Technology), Qianli Ma (South China University of Technology)
CodeRetrievalTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented Generation
π― What it does: Training large language models to enhance their discrimination and reasoning abilities regarding retrieved documents in the retrieval-augmented generation (RAG) task through self-correction learning.
π― What it does: This paper proposes a general method for training neural networks as recognizers of formal languages and provides an efficient length-constrained sampling algorithm for regular languages.
π― What it does: This study investigates the impact of Lipschitz continuity on the transferability of attacks and proposes a new ensemble training method called LOTOS, which enhances the robustness of model ensembles through hierarchical orthogonalization.
Training-Free Activation Sparsity in Large Language Models
James Liu (Massachusetts Institute of Technology), Ben Athiwaratkun (Together AI)
CodeTransformerLarge Language ModelText
π― What it does: We propose TEAL, a training-independent magnitude pruning activation sparsity method that can achieve 40-50% full model sparsity on modern LLMs;
Training-Free Diffusion Model Alignment with Sampling Demons
Po-Hung Yeh (Academia Sinica), Jun-cheng Chen
CodeGenerationOptimizationVision Language ModelDiffusion modelImageTextStochastic Differential EquationOrdinary Differential Equation
π― What it does: A training-free, non-backpropagation sampling method called Demon is proposed, which optimizes the noise in the reverse-time SDE randomly to align the diffusion model with user preferences.
CodeClassificationAnomaly DetectionTransformerLarge Language ModelTextTime Series
π― What it does: A training-independent LLM text generation detection method called Lastde/Lastde++ is proposed, which identifies the differences between human writing and model-generated text through time series analysis of token probability sequences.
Trajectory-LLM: A Language-based Data Generator for Trajectory Prediction in Autonomous Driving
Kairui Yang (Tianjin University), Di Lin (Shanghai Jiaotong University)
CodeGenerationData SynthesisAutonomous DrivingTransformerLarge Language ModelTextMultimodality
π― What it does: A vehicle trajectory generator called Traj-LLM based on large language models has been designed, which can automatically generate realistic, controllable, and diverse vehicle trajectories based on brief interactive descriptions, and has created the L2T dataset containing 240k pieces of interactive text, behaviors, and trajectories.
Transformer Block Coupling and its Correlation with Generalization in LLMs
Murdock Aubry (University of Toronto), Vardan Papyan (University of Toronto)
CodeTransformerLarge Language ModelText
π― What it does: A linearization analysis of the Jacobian matrix of large language model (LLM) Transformer blocks is conducted to study the relationship between the coupling of singular vectors between blocks and the model's generalization performance.
Transformer Meets Twicing: Harnessing Unattended Residual Information
Laziz Abdullaev, Tan Minh Nguyen
CodeRecognitionSegmentationTransformerImageText
π― What it does: A new Twicing Attention mechanism is proposed to replace traditional self-attention, enhancing the representational diversity of Transformers and mitigating the over-smoothing problem through residual self-correction.
CodeTransformerLarge Language ModelReinforcement LearningPrompt EngineeringMixture of ExpertsText
π― What it does: Proposes the Transformer 2 framework, utilizing Singular Value Fine-Tuning (SVF) to dynamically adapt large language models through two passes during inference;
Transformers Learn Low Sensitivity Functions: Investigations and Implications
Bhavya Vasudeva (University of Southern California), Vatsal Sharan (University of Southern California)
CodeTransformerImageTextMultimodality
π― What it does: This paper studies the low sensitivity function learned by Transformers in multimodal tasks and extends the sensitivity metric to non-Bool data, exploring its impact on robustness, loss landscape flatness, and training dynamics (grokking).
CodeClassificationRecognitionTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: A Tree of Attributes Prompt Learning (TAP) is constructed, embedding structured knowledge into the prompt learning of VLM through a hierarchical attribute tree generated by LLM;
π― What it does: By mapping the spatial state of PDEs to low-dimensional features extracted from scattering transforms and fitting with parameterized neural ODEs, predictable parameterized effective dynamics were learned, achieving bifurcation localization in noisy environments.
Trivialized Momentum Facilitates Diffusion Generative Modeling on Lie Groups
Yuchen Zhu (Georgia Institute of Technology), Molei Tao (Georgia Institute of Technology)
CodeGenerationData SynthesisDiffusion modelScore-based ModelBiomedical Data
π― What it does: A Lie group-based non-approximating diffusion generative model TDM is proposed, which transforms the manifold problem into score learning in Euclidean space using trivialized momentum, achieving efficient generation of manifold data.
π― What it does: This paper proposes a method to enhance trustworthy multi-view classification (TEF) through evolutionary multi-view fusion, which automatically searches for high-quality pseudo-views and addresses the information imbalance problem through view enhancement.
TS-LIF: A Temporal Segment Spiking Neuron Network for Time Series Forecasting
FENG SHIBO, Zhiqi Shen (University of Chinese Academy of Sciences)
CodeSpiking Neural NetworkTime Series
π― What it does: A dual-chamber pulse neuron model for time series prediction, TS-LIF, is proposed, utilizing two frequency processing mechanisms in dendrites and the soma to achieve multi-scale information extraction.
π― What it does: By migrating the CLIP text encoder from absolute position encoding to relative position encoding (RoPE), support for long texts (over 77 words) is achieved, forming the TULIP model.
CodeGenerationTransformerLarge Language ModelTextChain-of-Thought
π― What it does: The minp sampling method is proposed, which dynamically adjusts the sampling threshold using model confidence, thereby balancing diversity and coherence in LLM text generation.
π― What it does: This paper proposes a training-free two-stage sampling framework (TweedieMix) for synthesizing multi-concept images and videos during the inference phase of diffusion models.
Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models
Simon Schrodi (University of Freiburg), Thomas Brox (University of Freiburg)
CodeClassificationRetrievalVision Language ModelContrastive LearningImageMultimodality
π― What it does: Analyze and compare the relationship between modality gap, target bias, and information imbalance in contrastive vision models, and verify that information imbalance is the fundamental cause of both;
CodeCompressionOptimizationConvolutional Neural NetworkTransformerLarge Language ModelImageText
π― What it does: The Double Sparse Factorization (DSF) algorithm is proposed, which decomposes the weight matrix into two sparse matrices for a round of hierarchical sparsification;
TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking
Danqing Wang (Carnegie Mellon University), Lei Li (Qwen Team)
CodeTransformerLarge Language ModelTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Designed and implemented the TypedThinker framework, which utilizes a meta-thinker to predict suitable reasoning types and diversifies the reasoning process of LLM through explicit demonstration retrieval.
U-shaped and Inverted-U Scaling behind Emergent Abilities of Large Language Models
Tung-Yu Wu (National Taiwan University), Melody Lo (National Taiwan University)
CodeTransformerLarge Language ModelText
π― What it does: This study investigates the phenomenon of generative capabilities of large language models in multiple-choice tasks, finding that hard questions exhibit a U-shaped curve while easy questions show an inverted U-shaped curve. It explains the reasons for the performance stagnation followed by a sudden increase and proposes the Slice-and-Sandwich prediction pipeline.
UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models
Xin Xu (Hong Kong University of Science and Technology), Can Yang (Hong Kong University of Science and Technology)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: This paper presents UGMathBench, a dynamic benchmark containing 5,062 undergraduate-level math problems, covering 16 subjects, 111 topics, and 10 types of answers, with three random versions for each problem.
π― What it does: Using a diffusion model for two-stage sampling and attention injection, we generate synthetic faces that retain identity information while exhibiting high intra-class diversity, aimed at training face recognition models.
π― What it does: This paper proposes a framework for sparse 3D shape reconstruction and uncertainty estimation based on Dropsembles, which uses high-quality synthetic data as prior information and fine-tunes implicit functions under sparse noisy input.
Uncertainty-Aware Decoding with Minimum Bayes Risk
Nico Daheim (Technical University of Darmstadt), Iryna Gurevych (Technical University of Darmstadt)
CodeGenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes a language model decoding method that considers weight uncertainty by integrating parameter posteriors into minimum Bayes risk (MBR) decoding, reducing hallucinations and errors in generation, and achieving performance improvements without additional inference overhead.
Uncovering Gaps in How Humans and LLMs Interpret Subjective Language
Erik Jones (University of California Berkeley), Jacob Steinhardt (University of California Berkeley)
CodeTransformerLarge Language ModelText
π― What it does: This study investigates the discrepancies between the operational semantics of LLMs when processing subjective natural language and human expectations, and proposes a dictionary comparison-based error detection method.
π― What it does: A framework based on underdamped diffusion bridges is proposed, achieving finite-time convergence from prior to target distribution.
Understanding and Enhancing Safety Mechanisms of LLMs via Safety-Specific Neuron
Yiran Zhao (National University of Singapore), Michael Shieh
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes a method for the detection and tuning of safe neurons, identifying safe neurons that account for less than 1% of the parameters and enhancing the safety of LLMs by updating only these neurons.
Understanding and Enhancing the Transferability of Jailbreaking Attacks
Runqi Lin (Sydney AI Centre, University of Sydney), Tongliang Liu (Sydney AI Centre, University of Sydney)
CodeAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper studies the transferability of jailbreak attacks from the perspective of intent perception in large language models and proposes an attack method based on Intent Importance Flattening (PiF), significantly enhancing the destructive effect on proprietary models.
π― What it does: The bottlenecks of the Structured State Space Model (SSM) were studied, revealing the presence of asymptotic bias and over-smoothing issues, and a polarization technique was proposed to address them.
Understanding and Mitigating Hallucination in Large Vision-Language Models via Modular Attribution and Intervention
Tianyun Yang (Institute of Computing Technology, Chinese Academy of Sciences), Chang Xu (University of Sydney)
CodeGenerationExplainability and InterpretabilityTransformerVision Language ModelImageMultimodality
π― What it does: A causal attribution and intervention study on the hallucination problem of large visual language models, identifying specific heads in multi-head attention that cause hallucinations, and proposing two methodsβdeactivation and fine-tuning during decodingβto mitigate hallucinations.
Understanding Constraint Inference in Safety-Critical Inverse Reinforcement Learning
Bo Yue (Chinese University of Hong Kong), Guiliang Liu (Chinese University of Hong Kong)
CodeOptimizationSafty and PrivacyReinforcement LearningSequential
π― What it does: This paper compares two methods in safety-critical inverse reinforcement learning: Inverse Reward Correction (IRC) with implicit constraint embedding and Inverse Constraint Reinforcement Learning (ICRL) with explicit constraint inference. It provides an analysis of theoretical sample complexity and cross-environment transfer, and validates the results through grid world experiments.
Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry
Ziheng Chen (University of Trento), Nicu Sebe (University of Trento)
CodeClassificationImage
π― What it does: Explores and unifies the explanation of the roles of matrix logarithm and power functions in Global Covariance Pooling (GCP), clarifying their implicit construction of SPD polynomial Logistic regression (Riemannian classifier).
π― What it does: A continuous dynamic model named Central Flow is proposed and validated to describe the time-averaged optimization trajectory of gradient descent and adaptive optimizers (such as Scalar RMSProp and RMSProp) in the 'Edge of Stability' state in deep learning. By using third-order Taylor expansion and time averaging, the corresponding ordinary differential equations are derived and experimentally validated across various network architectures.
Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study
Xingxuan Zhang (Tsinghua University), Peng Cui (Tsinghua University)
CodeTransformerLarge Language ModelText
π― What it does: This paper systematically evaluates the generalization ability of Transformers in In-Context Learning (ICL), constructing a task-based three-dimensional framework (inter-question, intra-question, intra-task), and explores its generalization performance across different dimensions through experiments such as function fitting, tool invocation, and translation.
π― What it does: This paper conducts an algorithm-dependent generalization analysis and excess risk assessment of partially personalized federated learning (PFL) under non-convex conditions for the first time using uniform stability methods.
π― What it does: This paper proposes a unified and general multi-dataset 3D detection framework called Uni 2 Det, which utilizes a multi-stage prompting module to achieve feature unification and sharing across different datasets.
UniCO: On Unified Combinatorial Optimization via Problem Reduction to Matrix-Encoded General TSP
Wenzheng Pan (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
CodeOptimizationTransformerReinforcement LearningDiffusion model
π― What it does: Various combinatorial optimization problems are reduced to a general TSP matrix form in polynomial time, and a neural TSP solver capable of cross-task and cross-scale training is developed.
UniCoTT: A Unified Framework for Structural Chain-of-Thought Distillation
Xianwei Zhuang (Peking University), Yuexian Zou (Peking University)
CodeKnowledge DistillationTransformerLarge Language ModelContrastive LearningTextChain-of-Thought
π― What it does: By constructing and unifying diverse structured chains-of-thought (UniCoT) such as chain, tree, and graph structures, and using it as a bridge to transfer the reasoning capabilities of large models (LLM) to small models (SLM).
UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation
Huimin LU, Ichiro Sakata (University of Tokyo)
CodeKnowledge DistillationData-Centric LearningTransformerLarge Language ModelContrastive LearningText
π― What it does: We propose UNIDETOX, a knowledge distillation technique implemented through contrastive decoding, to generate detoxified text that can be fine-tuned on any large language model, achieving a unified detoxification effect.
Chenlu Ding (Hong Kong Polytechnic University), Xiangnan He (University of Science and Technology of China)
CodeRecommendation SystemOptimizationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes a unified parameter-efficient model forgetting framework called LLMEraser, which enables rapid parameter adjustment for instance-level forgetting tasks of LLMs (instance deletion, query modification, answer correction);
π― What it does: Designed and unified a modular ML4TSP framework, systematically breaking down and evaluating the roles of learning and search in TSP solving, and based on this, proposed improvement and restructuring methods.
Unifying Causal Representation Learning with the Invariance Principle
Dingling Yao (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
CodeRepresentation LearningAuto EncoderTabular
π― What it does: A unified causal representation learning framework is proposed, utilizing the invariance principle to achieve the identifiability of latent variables.
π― What it does: A unified benchmark for unsupervised graph-level anomaly detection and graph-level out-of-distribution (OOD) detection (UB-GOLD) has been constructed, and a systematic evaluation of 18 mainstream methods has been conducted on 35 multi-scenario datasets.
π― What it does: A unified diffusion model, UniGEM, has been designed and implemented to simultaneously perform molecular generation and molecular property prediction.
π― What it does: The UniGS framework is proposed, which for the first time uses 3D Gaussian Splatting (3DGS) as a unified tri-modal (text-image-3D) pre-trained representation, addressing the sparsity of point cloud representation and the continuity gap with images.
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
Noam Razin (Princeton University), Boris Hanin (Princeton University)
CodeOptimizationSafty and PrivacyReinforcement Learning from Human FeedbackText
π― What it does: This paper studies the phenomenon of 'likelihood shift' in Direct Preference Optimization (DPO, etc.), elucidating the underlying geometric mechanisms of embeddings. It proposes and validates the Centralized Hidden Embedding Similarity (CHES) score, which is used to identify and filter training samples that lead to severe shifts, thereby alleviating unintentional mismatches in the model's safety alignment tasks.
π― What it does: This study investigates the transfer performance of pre-trained models in infrared image semantic segmentation, systematically comparing various pre-training methods, and proposes a unified pre-training framework called UNIP based on attention pattern analysis.
π― What it does: A Degradation Classification Pre-Training (DCPT) framework is proposed, allowing the restoration network to first learn to identify image degradation types through pre-training before downstream tasks, thereby enhancing global restoration capabilities.
π― What it does: Research on the security of machine unlearning systems demonstrates that submitting adversarial unlearning requests leads to a sharp decline in model accuracy.
Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment
Yankai Jiang (Shanghai AI Laboratory), Shaoting Zhang (Shanghai AI Laboratory)
CodeSegmentationTransformerVision Language ModelContrastive LearningImageMultimodalityBiomedical DataMagnetic Resonance Imaging
π― What it does: This paper presents Malenia, a multi-scale mask-attribute alignment framework for 3D zero-shot lesion segmentation, and designs a cross-modal knowledge injection (CMKI) module.
Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues
Riccardo Grazzi (Istituto Italiano di Tecnologia), Massimiliano Pontil
CodeRecurrent Neural NetworkTextSequential
π― What it does: This paper studies the expressive power of Linear Recursive Neural Networks (LRNN) in state tracking tasks, proving that using only positive eigenvalues leads to the inability to solve problems such as parity and modular arithmetic. By extending the range of the eigenvalues of the state transition matrix to [-1, 1], the model's expressiveness is significantly enhanced, achieving perfect parity and significantly improved performance in modular arithmetic, word formation problems, and code/mathematical language modeling in multiple experiments.
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning
Gangwei Jiang (University of Science and Technology of China), Ying Wei (Zhejiang University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper analyzes catastrophic forgetting in continuous instruction fine-tuning of LLMs through functional vector analysis and proposes a functional vector-guided training method to alleviate this issue.
π― What it does: This paper proposes a complete pipeline for indoor layout estimation based on Plane-DUSt3R, capable of directly generating the 3D layout of walls, ceilings, and floors from multiple perspective images without pose constraints.