International Conference on Machine Learning · 1032 papers
TEAM: Temporal–Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration
Linye Wei (Peking University), Meng Li (Peking University)
CodeComputational EfficiencyAI Code AssistantTransformerLarge Language ModelMixture of ExpertsDiffusion modelText
🎯 What it does: Proposed the TEAM framework, which utilizes spatiotemporal consistency to accelerate Mixture-of-Experts (MoE) diffusion large language models (dLLM)
Tempora: Characterising the Time-Contingent Utility of Online Test-Time Adaptation
Sudarshan Sreeram (University of Cambridge), Cecilia Mascolo (University of Cambridge)
CodeClassificationDomain AdaptationExplainability and InterpretabilityComputational EfficiencyConvolutional Neural NetworkTransformerSupervised Fine-TuningContrastive LearningImage
🎯 What it does: Introduce the Tempora framework, systematically evaluate the effectiveness of test-time adaptation (TTA) under different time pressures, and define three time-related utility metrics (discrete, continuous, amortized)
🎯 What it does: Propose the TEP-SAM framework, which utilizes gradually emerging target motion information over time. By employing a Discrepancy-Enhanced Temporal Encoder and a Temporal Prompt Generator, temporal features are injected into SAM to achieve interactive-free multi-frame infrared small target detection and segmentation.
Test-Time Debiasing with Probabilistic Prompts via Wasserstein Distance in Vision-Language Models
Chengye Wang (Zhejiang University), Chaochao Chen (Zhejiang University)
CodeClassificationRetrievalDomain AdaptationExplainability and InterpretabilityTransformerPrompt EngineeringVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Propose a distributed debiasing framework called W4D based on the Wasserstein distance, which achieves fair query embedding through parameter-free updates at test time using probabilistic prompts.
CodeGenerationExplainability and InterpretabilityComputational EfficiencyData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a detoxification method that is performed only during the inference phase, utilizing zeroth-order gradient descent to optimize input word embeddings, thereby reducing the toxicity of generated text without training the model or modifying its parameters.
🎯 What it does: This paper proposes an online distribution drift detection method based on Conditional Conformal Test Martingale (CCTM), aiming to avoid the test-time contamination problem that occurs in traditional CTM methods;
CodeOptimizationComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Proposes TetraJet-v2, an end-to-end 4-bit full quantization training (FQT) method for LLMs, using the NVFP4 format for quantizing activations, weights, and gradients.
Text-Driven Fusion for Infrared and Visible Images: Achieving Image Scene Adaptation on Hyperbolic Space
Huan Kang (Jiangnan University), Josef Kittler (University of Surrey)
CodeImage TranslationRestorationDomain AdaptationTransformerPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Proposed the TEDFusion framework, which utilizes text-driven hyperbolic space for infrared and visible image fusion, achieving adaptive fusion without text input during inference.
TG-RAG: A Retrieval-Augmented Framework for Reasoning Guidance in Specialized Domains
Liang Su (Fudan University), Li Sun (Beijing University of Posts and Telecommunications)
CodeRecommendation SystemExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextGraphTabularFinance RelatedRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed a thought-guided framework based on retrieval augmentation called TG-RAG, which structures standard operating procedures (SOP) using an Expert Procedure Graph (EPG), and dynamically injects step instructions during the reasoning process through an Interrupt-Retrieve-Generate (IRG) mechanism, achieving active control over large reasoning models.
The Appeal and Reality of Recycling LoRAs with Adaptive Merging
Haokun Liu (University of Toronto), Colin Raffel (University of Toronto)
CodeComputational EfficiencyKnowledge DistillationRepresentation LearningData-Centric LearningTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsTextBenchmark
🎯 What it does: Evaluated the impact of using adaptive merging methods on multi-task performance across approximately 1000 LoRA pools contributed by real users.
The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level
Jeremy Herbst (University of Hamburg), Jae Hee Lee (University of Hamburg)
CodeExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsContrastive LearningTextSequential
🎯 What it does: This paper investigates the interpretability of Mixture-of-Experts (MoE) language models, comparing the polysemy of MoE experts and dense FFN neurons using k-sparse probes, and demonstrates that the expert level can serve as an interpretable module through LLM-generated expert function labels and causal validation.
The Expressive Power of Low Precision Softmax Transformers with (Summarized) Chain-of-Thought
Moritz Brösamle (University of Tübingen), Stephan Eckstein (University of Tübingen)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: This paper proves that under low-precision activation and attention weight rounding, the standard softmax Transformer can achieve Turing completeness through Chain-of-Thought and Summarized Chain-of-Thought, and provides specific bounds on how depth and width grow logarithmically with context length.
The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning
Will Hawkins (Oxford Internet Institute, University of Oxford), Chris Russell (Oxford Internet Institute, University of Oxford)
CodeSafty and PrivacyExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper systematically evaluates the impact of non-attackive data translation on model safety behavior by performing LoRA fine-tuning on the Llama-3.2, Qwen3, and Gemma-3 models across multiple languages.
The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation
Akanksha Narula (Max Planck Institute for Software Systems), Laurent Bindschaedler (Max Planck Institute for Software Systems)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Studies how prompt context influences large language models' selection of different algorithms in code generation, leading to performance and safety differences in the implementation of the same functionality.
The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models
Zheng Wang (Bosch Center for Artificial Intelligence), Xiaonan Lu (Bosch Center for Artificial Intelligence)
CodeOptimizationExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextSequentialBenchmark
🎯 What it does: This paper studies sequential knowledge editing in large language models, proposes a theoretical framework that establishes the equivalence between single-edit and sequential editing, reveals the fundamental reasons behind editing stability, and systematically evaluates common regularization techniques.
The Perception–Physics Paradox: Probing Scientific Alignment with TC-Bench
Dingling Yao (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
CodeExplainability and InterpretabilityTransformerVision Language ModelAuto EncoderContrastive LearningImageTabularBenchmarkPhysics Related
🎯 What it does: This paper introduces the concept of Scientific Alignment, defines Structural Isomorphism, and constructs the TC-BENCH global tropical cyclone benchmark dataset based on this, aiming to systematically explore the interpretability and alignment of visual foundation models (VFM) in satellite images with respect to physical states, revealing the contradiction between perception and physical reasoning.
The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning
Haolong Qian (Tsinghua University), Chun Yuan (Tsinghua University)
CodeComputational EfficiencyKnowledge DistillationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Studies the 'quality-utility paradox' that occurs when using a strong teacher model to generate or revise data for small model mathematical reasoning tasks, and proposes a revision method based on style alignment to reduce adaptation costs and improve downstream performance.
The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models
Liuyuan Wen (Nanjing University), Yang Gao (Nanjing University)
CodeExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Investigate the geometric structure of residual flows in large language models on multi-digit addition tasks, identifying and explaining the Iso-Raw-Sum Trajectory (IRST) and its resulting arithmetic errors
🎯 What it does: To address the Andrews-Curtis conjecture in mathematical reasoning, it is modeled as a reinforcement learning search problem with sparse rewards, and three innovative approaches are proposed: data generation, algorithm improvement, and model architecture, successfully solving over 100 previously unsolved examples.
Yiping Wang (University of Washington), yelong shen
CodeOptimizationMeta LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextChain-of-Thought
🎯 What it does: Developed the ThetaEvolve framework, which enables a single LLM to perform reasoning and reinforcement learning on open mathematical optimization problems, continuously improving programs and surpassing existing optimal bounds during testing.
CodeImage TranslationGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringDiffusion modelImageTextMultimodalityChain-of-Thought
🎯 What it does: Proposed and implemented the 'Think‑Then‑Generate' (T2G) paradigm, enabling the LLM encoder to first reason and rewrite the original text prompt, and then embed the rewritten prompt into a diffusion model for image generation.
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodalityBenchmark
🎯 What it does: Propose the GeoThinker framework, which actively injects 3D geometric information according to task requirements into a multi-modal large language model to enhance spatial reasoning capabilities.
Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space
Houjun Liu (Stanford University), Róbert Csordás (Stanford University)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: Proposes an unsupervised Transformer variant called Thoughtbubbles, which improves reasoning efficiency without additional supervision by performing dynamic forking and removing residual flows in hidden layers, enabling parallel adaptive computation in the latent space.
Threshold-Based Exclusive Batching for LLM Inference
Weifang Zhang (Hong Kong Polytechnic University), Shining Wu (Hong Kong Polytechnic University)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose a threshold-based exclusive batching (EB) scheduling strategy, and design an online adaptive controller and a hybrid scheduler EB+, to optimize the throughput of LLM inference under different GPU bandwidths, model scales, and workload mixing ratios.
CodeOptimizationTransformerSupervised Fine-TuningContrastive LearningTabularTime SeriesBenchmarkPhysics Related
🎯 What it does: This paper systematically analyzes time series forecasting models from a dynamical perspective, proposing the PRO-DYN methodology, which decomposes models into three components: preprocessing (PRO), dynamics (DYN), and postprocessing (PRO). Within this framework, the paper investigates the key factors affecting model performance.
Time-Conditioned Foreseeing: An EHR-Specific Foundation Model for Irregular Dynamics and Calendrical Time
Bong Gyun Kang (Seoul National University), Sungroh Yoon (Seoul National University)
CodeTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsContrastive LearningTabularTime SeriesBiomedical DataElectronic Health Records
🎯 What it does: Designed and trained a foundation model for EHR, TCF PFM, which can accurately process irregular time and numerical data, supporting event generation and prediction with multiple time windows.
TiME: Test-Time Mixture-of-Experts Routing via Asymmetric CO-Optimal Transport for Continual Test-Time Adaptation
Tianlun Liu (National University of Defense Technology), Dongsheng Li (National University of Defense Technology)
CodeDomain AdaptationComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerMixture of ExpertsContrastive LearningTextBiomedical DataBenchmark
🎯 What it does: Propose an online routing and parameter update strategy (TiME) for Mixture-of-Experts (MoE) models in Continuous Testing Time Adaptation (CTTA) environments.
TimeOmni-VL: Unified Models for Time Series Understanding and Generation
Tong Guan (Griffith University), Shirui Pan (Griffith University)
CodeGenerationData SynthesisAnomaly DetectionRepresentation LearningTransformerLarge Language ModelVision Language ModelDiffusion modelContrastive LearningImageTime SeriesChain-of-Thought
🎯 What it does: A unified time series model called TIMEOMNI-VL was constructed, which can simultaneously perform semantic understanding of time series (such as question answering, pattern analysis) and numerical generation (such as prediction, interpolation).
TN-SHAP-G: Graph-Structured Tensor Network Surrogates for Shapley Values and Interactions
Farzaneh Heidari (Universit´ e de Montr´ eal), Guillaume Rabusseau (Universit´ e de Montr´ eal)
CodeExplainability and InterpretabilityDrug DiscoveryGraph Neural NetworkGraphBenchmark
🎯 What it does: Construct a graph-structured aligned tensor network (TN) surrogate model to learn the multilinear extension of masked graph games from limited black-box model queries, then deterministically recover Shapley values and their interaction terms directly through polynomial interpolation and closed-form integration.
Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
Dongwon Jo (Seoul National University), Jae-Joon Kim (Seoul National University)
CodeComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Proposes Token Sparse Attention, a dynamic and reversible token-level sparse attention mechanism, which maintains attention to all tokens by compressing Q/K/V at each layer and decompressing back to the full sequence.
CodeOptimizationFederated LearningComputational EfficiencyAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningMixture of ExpertsTextBenchmark
🎯 What it does: Proposed the FUSIONROUTE framework, which实现了 token-level selection of specialized LLMs at each decoding step, and provided complementary logits for correction through a lightweight router;
Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach
Sicheng Liu (Beijing Institute of Technology), Guoren Wang (Beijing Institute of Technology)
CodeClassificationGenerationComputational EfficiencyRepresentation LearningGraph Neural NetworkTransformerMixture of ExpertsAuto EncoderContrastive LearningImageTextMultimodalityGraph
🎯 What it does: Designed and implemented a foundation model called PLANET for multi-modal graphs, addressing modality interaction and alignment issues through a hierarchical strategy, supporting node classification, link prediction, and generation tasks from graphs to text/images.
Toward Scalable and Valid Conditional Independence Testing with Spectral Representations
Alek Fröhlich (Istituto Italiano di Tecnologia), Massimiliano Pontil (Istituto Italiano di Tecnologia)
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningContrastive LearningImageTabularBiomedical DataBenchmark
🎯 What it does: Proposed a scalable conditional independence test method called SpectralCIT based on spectral representation learning, which constructs a simple test statistic using the low-rank singular value decomposition of the partial correlation operator, and provides theoretical guarantees for type I error and power.
Toward Training Superintelligent Software Agents through Self-Play SWE-RL
Yuxiang Wei (Meta FAIR), Sida Wang (Meta FAIR)
CodeAutonomous DrivingOptimizationFederated LearningComputational EfficiencyData-Centric LearningRobotic IntelligenceDrug DiscoveryAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringDiffusion modelWorld ModelTextSequentialBenchmark
🎯 What it does: Through self-supervised training, LLMs autonomously generate and fix bugs in warehouse environments without human-labeled data, achieving continuous self-improvement of software agents.
Towards a Holistic Understanding of Selection Bias for Causal Effect Identification
Yiwen Qiu (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
CodeOptimizationFederated LearningExplainability and InterpretabilityComputational EfficiencyData-Centric LearningTabularBiomedical DataElectronic Health Records
🎯 What it does: A unified framework based on distribution classes is proposed, providing necessary and sufficient conditions for the identifiability of the average treatment effect under selection bias, achieving identifiability across various selection mechanisms and distribution families.
🎯 What it does: Designed a complete partitioning strategy for parameter sharing in multi-agent reinforcement learning, combined with a diffusion model to achieve the Dspic algorithm, enabling efficient collaborative learning among multiple agents.
🎯 What it does: Proposes the FAN algorithm, combining the regularization of one-time flow iteration behavior with single-noise distributed Q-learning, achieving efficient and expressive performance in offline reinforcement learning.
Towards Feedback-to-Plan Decisions for Self-Evolving LLM Agents in CUDA Kernel Generation
Yee Hin Chong (Tsinghua University), Peng Qu (Tsinghua University)
CodeOptimizationExplainability and InterpretabilityAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Propose the CUDAnalyst framework, which performs causal attribution at the generation level for feedback-to-plan decision-making in self-evolving LLM agents, adopting trajectory freezing, feedback injection, and Banzhaf cooperative attribution;
Towards Functional Correctness of Large Code Models with Selective Generation
Jaewoo Jeong (POSTECH), Sangdon Park (POSTECH)
CodeGenerationAI Code AssistantTransformerLarge Language ModelReinforcement LearningPrompt EngineeringDiffusion modelTextSequential
🎯 What it does: Propose a controllable selective code generation method, which uses automatically generated unit tests (via fuzz testing) to evaluate the functional correctness of the code, and self-abandons during the generation process based on confidence levels, thereby controlling the functional hallucination rate in code generation.
Towards High-Fidelity CAD Generation via LLM-Driven Program Generation and Text-Based B-Rep Primitive Grounding
Jiahao Li (Fudan University), Xiangdong Zhou (Fudan University)
CodeGenerationData SynthesisGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextMeshGraph
🎯 What it does: Propose the FutureCAD framework, which utilizes LLM to generate executable CadQuery scripts and precisely locate B-Rep geometric primitives through text queries, thereby enabling the automatic generation of high-fidelity CAD models.
Towards Multimodal Large Language Models with Both Training and Inference Efficiency
Qianhao Yuan (Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences), Le Sun
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelContrastive LearningImageVideoTextMultimodality
🎯 What it does: This paper proposes a novel multimodal large language model architecture called SAISA, aiming to simultaneously improve training efficiency and inference efficiency.
CodeKnowledge DistillationData-Centric LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningContrastive LearningTextBenchmark
🎯 What it does: Proposes theoretical and methodological approaches for aligning the model's own distribution in supervised fine-tuning (SFT), utilizing distribution discrimination theory (DDT) to construct a centralized log-likelihood (CLL) discrimination metric, and further designing adaptive loss reweighting techniques IDFT and entropy-guided hybrid decoding methods Hinted Decoding;
Towards Solving the Gilbert-Pollak Conjecture via Large Language Models
Yisi Ke (Peking University), Liwei Wang (Peking University)
CodeSymbolic ComputationOptimizationComputational EfficiencyKnowledge DistillationAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextReview/Survey PaperPhysics Related
🎯 What it does: Using large language models (LLMs) to automatically generate structured geometric lemmas and convert them into verifiable functions to iteratively improve the lower bound of the Steiner ratio
Towards the Explainability of Temporal Graph Networks via Memory Backtracking and Topological Attribution
Yazheng Liu (Hong Kong University of Science and Technology), Hui Xiong (Hong Kong University of Science and Technology)
CodeExplainability and InterpretabilityGraph Neural NetworkGraphTime Series
🎯 What it does: Study the explainability of Temporal Graph Networks, proposing to interpret prediction results through memory backtracking and topological attribution trees.
Magamed Taimeskhanov (University of Würzburg), Damien Garreau (Université Côte d'Azur)
CodeSafty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Controlling the behavior of large language models (LLMs) during inference by linearly shifting (steering) the intermediate representations, and theoretically analyzing the steering strength α.
Towards Whole-corpus Reconstruction of Heterogeneous RAG Knowledge Bases
Peiru Yang (Beijing University of Posts and Telecommunications), Tao Qi (Beijing University of Posts and Telecommunications)
CodeRetrievalRepresentation LearningAdversarial AttackTransformerLarge Language ModelContrastive LearningGaussian SplattingTextMultimodalityBenchmarkFinance RelatedRetrieval-Augmented Generation
🎯 What it does: Propose the GeoEx framework, which utilizes a proxy encoder to plan queries in the embedding space and generates natural language queries through embedding inversion, achieving a full reconstruction of multi-source RAG knowledge bases without prior knowledge.
Shashwat Goel (ELLIS Institute Tübingen), Chenxi Whitehouse (Meta Superintelligence Labs)
CodeMeta LearningDrug DiscoveryAI Code AssistantReinforcement Learning from Human FeedbackNeural Architecture SearchTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed an scalable training process that utilizes automatically extracted research objectives and corresponding scoring rubrics, combined with reinforcement learning to enable language models to generate superior research plans.
🎯 What it does: This paper proposes a non-normalization deep spiking neural network training initialization framework called SpikeInit, addressing the dependencies and limitations of traditional BN in SNN training.
Training Prompt Matters: State-Adaptive Optimization for Robust Fine-Tuning
Wenhang Shi (Renmin University of China), Xiaoyong Du (Renmin University of China)
CodeOptimizationFederated LearningExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationRepresentation LearningAdversarial AttackHyperparameter SearchData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: In the process of fine-tuning large language models, the impact of prompt selection during the training phase on model performance was systematically studied, and a dynamic prompt optimization method based on pre-updated loss (SAPO) was proposed. By generating multiple semantically equivalent prompts and selecting the one with the lowest pre-updated loss as the training prompt, this method significantly reduces forgetting and enhances generalization.
🎯 What it does: Proposes a particle filtering method that requires no additional training, utilizing a diffusion model generator to directly achieve the optimal proposal distribution, addressing the integration of Kalman filtering and particle filtering in high-dimensional systems.
Training-Free Coverless Multi-Image Steganography with Access Control
Minyeol Bae (Korea Advanced Institute of Science and Technology), Si-Hyeon Lee (Korea Advanced Institute of Science and Technology)
CodeSafty and PrivacyTransformerDiffusion modelAuto EncoderImage
🎯 What it does: Propose MIDAS, a training-agnostic image-coverless multi-image steganography framework that supports access control based on private keys;
Training-Free Hashing-Based Attention via Binary Principal Components
Daohai Yu (Xiamen University), Rongrong Ji (Xiamen University)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringAuto EncoderTextBenchmark
🎯 What it does: BinaryPC proposes a training-agnostic, data-aware hashing sparse attention mechanism, achieving efficient retrieval and decoding by compressing the KV cache using binary principal component analysis;
🎯 What it does: This paper proposes an untrained vector quantization method called Gaussian Quant (GQ), which first trains a constrained Gaussian Variational Autoencoder (Gaussian VAE), and then matches its posterior mean with a set of pre-generated Gaussian noise codebook, achieving a complete process of directly converting Gaussian VAE into VQ-VAE.
🎯 What it does: Propose a trajectory-based shortcut data augmentation method (LIFT), which improves data quality in offline reinforcement learning by inserting high-value actions during the logging policy collection process.
Transitivity Meets Cyclicity: Explicit Preference Decomposition for Dynamic Large Language Model Alignment
Yucong Huang (Harbin Institute of Technology), Jing Li (Harbin Institute of Technology)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringScore-based ModelTextBenchmarkChain-of-Thought
🎯 What it does: Proposed the Hybrid Reward-Cyclic (HRC) model, which explicitly decomposes human preferences into a transitive scalar reward and a cyclic vector component, and combines Dynamic Self-Play Preference Optimization (DSPPO) for time-varying game optimization.
Translation Heads: Disentangling meaning from language in LLM-based machine translation
Théo Lasnier (Inria), Benoît Sagot (Inria)
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextMultimodality
🎯 What it does: This paper analyzes the internal workings of LLMs in sentence-level machine translation from the perspective of mechanism interpretability, using activation patching to perform fine-grained localization of attention heads, and decomposing the translation task into two subtasks: target language identification and sentence equivalence.
Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometries
Hang Zhou (Tsinghua University), Mingsheng Long (Tsinghua University)
CodeOptimizationComputational EfficiencyGraph Neural NetworkTransformerPrompt EngineeringMixture of ExpertsDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningMeshGraphPhysics Related
🎯 What it does: Propose Transolver-3, a Transformer PDE solver designed for industrial-scale geometry (10⁸+ elements), capable of being trained on a single GPU and achieving high-fidelity field prediction during inference.
TreeCUA: Efficiently Scaling GUI Automation with Tree-Structured Verifiable Evolution
Deyang Jiang (Meituan), Zhixiong Zeng (Meituan)
CodeAutonomous DrivingOptimizationComputational EfficiencyRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringMixture of ExpertsVision-Language-Action ModelDiffusion modelWorld ModelTextSequentialRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: TreeCUA proposes a tree-structured, verifiable evolutionary multi-agent framework, achieving efficient synthesis of large-scale GUI automation trajectories.
🎯 What it does: Propose the TriForces three-stream framework and self-supervised pre-training to enhance the transferability and retrieval performance of MLIP.
TSFAdv: Frequency-Guided Black-Box Adversarial Attacks on Time Series Forecasting
Qizhuo Han (Nankai University), Zheli Liu (Nankai University)
CodeAdversarial AttackTime Series
🎯 What it does: Developed TSFAdv, a black-box attack framework based on frequency-domain sensitivity analysis and natural evolution strategies, capable of inducing significant errors in long-sequence prediction models within a limited number of queries.
TT-Sparse: Learning Sparse Rule Models with Differentiable Truth Tables
Hans Farrell Soegeng (Nanyang Technological University), Thomas Peyrin (Nanyang Technological University)
CodeClassificationOptimizationExplainability and InterpretabilityAuto EncoderContrastive LearningTabular
🎯 What it does: Designed and implemented TT-SPARSE, a sparse rule learning module based on differentiable truth tables, to generate globally interpretable and precise rule sets within a single network.
Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization
Chen Liang (Yale University), Daniel Rakita (Yale University)
CodeOptimizationImageTabular
🎯 What it does: Proposed a new zeroth-order optimization method called Coherent Coordinate Descent (CoCD), which achieves efficient deterministic coordinate descent by utilizing old gradients and implicit smoothing.
TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins
Nikita Makarov (Roche), Michael Patrick Menden (Helmholtz Munich)
CodeExplainability and InterpretabilityRepresentation LearningData-Centric LearningDrug DiscoveryReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextTabularTime SeriesBiomedical DataElectronic Health RecordsRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the TwinWeaver framework, which leverages LLM to serialize multimodal medical records into text, training the Genie Digital Twin (GDT) to achieve joint time series prediction and clinical event prediction across 20 types of cancer;
Two-dimensional quantization for geometry-aware audio coding
Tal Shuster (Ben Gurion University Of Negev), Eliya Nachmani (Ben Gurion University Of Negev)
CodeCompressionAuto EncoderAudio
🎯 What it does: This paper proposes a two-dimensional geometric quantization (Q2D2), which maps features to hexagonal, rectangular, or rhombic grids for quantization;
U-Cast: A Surprisingly Simple and Efficient Frontier Probabilistic AI Weather Forecaster
Salva Rühling Cachay (UC San Diego), Rose Yu (UC San Diego)
CodeOptimizationComputational EfficiencyRepresentation LearningAI Code AssistantConvolutional Neural NetworkTransformerMixture of ExpertsAuto EncoderContrastive LearningTabularTime Series
🎯 What it does: Developed a probabilistic weather forecasting model named U-Cast, which utilizes the standard U-Net architecture and achieves high-precision 15-day medium-range forecasts through a two-stage training process (pre-training with MAE, followed by fine-tuning with CRPS).
UAV$^2$: A Unified and Adaptive Scheduling Framework for UAV Autopilot System with Reinforcement Learning
Zeying Li (Sun Yat-sen University), Kai Huang (Sun Yat-sen University)
CodeAutonomous DrivingOptimizationRecurrent Neural NetworkTransformerReinforcement LearningSimultaneous Localization and MappingOptical FlowTime SeriesSequential
🎯 What it does: Propose a unified adaptive scheduling framework based on reinforcement learning, UAV 2, which integrates navigation and flight control into a single platform and learns task frequency scheduling through RL.
UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models
Jiaqi Wang (Beijing University of Posts and Telecommunications), Xinlong Wang (Beijing Academy of Artificial Intelligence)
CodeGenerationOptimizationReinforcement Learning from Human FeedbackTransformerReinforcement LearningPrompt EngineeringDiffusion modelScore-based ModelImageTextMultimodality
🎯 What it does: Proposed and implemented UDM-GRPO, which integrates the Uniform Discrete Diffusion model with Group Relative Policy Optimization (GRPO), for text-to-image generation tasks.
UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation
Danning Zhang (University of Science and Technology of China), Zhendong Mao (Institute of Artificial Intelligence Hefei Comprehensive National Science Center)
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Propose the UFO framework, which atomically chain-evaluates full conditional alignment in multi-modal image generation, and constructs the UFO-Bench benchmark.
CodeSegmentationReinforcement Learning from Human FeedbackTransformerReinforcement LearningPrompt EngineeringVision Language ModelContrastive LearningImageText
🎯 What it does: Proposes UGround, a unified visual orientation segmentation framework that dynamically selects intermediate layers in the unrolled Transformer as 'mask as prompt' and directly connects with SAM through skip-connection.
UHR-BAT: Budget-Aware Token Compression Vision-Language model for Ultra-High-Resolution Remote Sensing
Yunkai Dang (Nanjing University), Yang Gao (Nanjing University)
CodeCompressionComputational EfficiencyTransformerPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a budget-aware token compression framework called UHR-BAT for ultra-high-resolution remote sensing images, which can maintain query-related details and global context under strict visual token budgets;
Unbiased Alignment for Large Language Models with Noisy Preferences
Jialiang Wang (Harbin Institute of Technology), Haoliang Li (City University of Hong Kong)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringContrastive LearningText
🎯 What it does: This paper proposes a theoretical framework that can achieve unbiased alignment of large language models using only noisy preference data.
Uncovering Bias Mechanisms in Observational Studies
Ilker Demirel (Massachusetts Institute of Technology), David Sontag (Massachusetts Institute of Technology)
CodeFederated LearningExplainability and InterpretabilityData-Centric LearningSupervised Fine-TuningContrastive LearningOptical FlowTabularBiomedical DataElectronic Health RecordsReview/Survey PaperStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A diagnostic framework is constructed by linking the magnitude of bias in observational studies with the predictive performance of nuisance functions, aiming to reveal the mechanisms behind the bias.
Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference
Zifan He (University of California Los Angeles), Jason Cong (University of California Los Angeles)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: This paper unifies memory processing in LLM inference into a four-stage pipeline (Prepare Memory, Compute Relevance, Retrieve, Apply to Inference), revealing that it accounts for 22%-97% of inference latency, and proposes methods to accelerate different stages.
Understanding Data Temporality Impact on Large Language Models Pre-training
Romain Fabre (Kyutai), Edouard Grave (Kyutai)
CodeExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextTime SeriesSequentialBenchmarkRetrieval-Augmented Generation
🎯 What it does: The study investigates the impact of data order during pre-training on the acquisition of temporal knowledge in large language models, constructing a time-sensitive question-answering benchmark named KairosQA with over 7,000 items, and comparing pre-trained models of 6B parameters trained with ordered and shuffled data.
Unfolded Laplacian Spectral Embedding: A Theoretically Grounded Approach to Dynamic Network Representation
Haruka Ezoe (University of Tokyo), Ryohei Hisano (University of Tokyo)
CodeRepresentation LearningGraph Neural NetworkContrastive LearningGraphTime Series
🎯 What it does: Proposed a dynamic network embedding method called ULSE based on normalized Laplacian, and provided theoretical stability proof and experimental verification.
CodeGenerationExplainability and InterpretabilityComputational EfficiencyDiffusion modelFlow-based ModelImageStochastic Differential Equation
🎯 What it does: A global linearization framework based on the Koopman operator is constructed, performing a first-order linear approximation on the pre-trained Conditional Flow Matching (CFM) generative model, enabling its generation process to be analytically solved and sampled in one go.
🎯 What it does: Proposes an uncertainty-guided diffusion bridge model (UDBM) that solves the all-in-one image restoration (AiOIR) problem in a single-step reasoning manner.
UniRTL: Unifying Code and Graph for Robust RTL Representation Learning
Yi Liu (Chinese University of Hong Kong), Qiang Xu (Chinese University of Hong Kong)
CodeKnowledge DistillationRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelContrastive LearningTextMultimodalityGraph
🎯 What it does: Propose UniRTL, a multi-modal pre-training framework that unifies RTL code with complete control data flow graphs (CDFG) to obtain more robust RTL representations.
Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation
Jinyu Liu (Fudan University), Yu-Gang Jiang (Fudan University)
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: Design and release the Unison benchmark to evaluate the four-dimensional performance of unified multimodal models in collaborative capabilities of understanding and generation.
🎯 What it does: A new meta-learning theoretical framework is proposed, and under this framework, a Transformer-based algorithm implicit meta-learner called TAIL is designed to achieve generalization across domains, modalities, and label spaces in few-shot scenarios.
Universal Reasoner: A Single, Composable Plug-and-Play Reasoner for Frozen LLMs
Jaemin Kim (Korea Advanced Institute of Science and Technology), Jong Chul Ye (Korea Advanced Institute of Science and Technology)
CodeComputational EfficiencyKnowledge DistillationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextMultimodalityRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes Universal Reasoner (UniR), a lightweight and composable reasoning module that can be added to the logits of a frozen large language model (LLM);
Unlearning Isn’t Forgetting: Revealing Hidden Leakage in Class Unlearning Evaluations
Ali Ebrahimpour-Boroojeny (University of Illinois at Urbana-Champaign), Hari Sundaram (University of Illinois at Urbana-Champaign)
CodeClassificationSafty and PrivacyAdversarial AttackConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: Proposed a class-level forgetting method based on tilted reweighting (TREW) and designed a new class member inference attack (CMIA) to evaluate the forgetting effect.
Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization
Shiping Gao (Sun Yat-sen University), Lifu Huang (University of California Davis)
CodeOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextSequential
🎯 What it does: Propose the Implicit Prefix-Value Reward Model (IPVRM) and combine it with Distribution-Level RL (DistRL) to improve the quality of process rewards based on terminal correctness labels, and achieve more efficient sample utilization in RL.
🎯 What it does: In infrared target detection, the authors design and optimize a learnable Fourier shape attack, generating shapes that can be physically cut into thermal insulation materials, thereby making the target ignored by the detector in infrared images.
🎯 What it does: Proposes an ODE-driven continuous model merging framework, ODE-M, treating each model merge as a continuous trajectory in the parameter space rather than a one-time update.
Unsupervised Process-Aware Coreset Selection for In-Context Learning
Wei Zheng (Shandong University), YUQING SUN
CodeOptimizationRepresentation LearningData-Centric LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Proposes an unsupervised process-aware core set selection method (PaCS-ICL) for generating high-quality few-shot context demonstrations for large language models under extremely low labeling budgets.
USE: A Unified Self-Ensembling Framework for Test-Time Prompt Tuning
Siru Jiang (University of Chinese Academy of Sciences), Tieniu Tan (University of Chinese Academy of Sciences)
CodeClassificationDomain AdaptationOptimizationComputational EfficiencyTransformerPrompt EngineeringVision Language ModelAuto EncoderContrastive LearningImageTextMultimodality
🎯 What it does: Propose a unified self-integration framework called USE, which employs a self-integration strategy (SE) to perform self-supervised optimization on text prompts during testing, and uniformly applies the same strategy during inference to obtain more reliable pseudo-labels;
Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning
Heming Zou (Tsinghua University), Xiangyang Ji (Tsinghua University)
CodeComputational EfficiencyData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: A framework for online batch selection (UDS) oriented towards supervised fine-tuning of large language models was researched and implemented, aiming to improve efficiency and effectiveness by dynamically evaluating and filtering training samples.
🎯 What it does: Proposes the Variance Driven Exploration (VarDE) method for sample allocation in pure exploration scenarios, with the core idea of minimizing the uncertainty of the final decision by multiplying the sensitivity weight of the decision function with the local variance;
🎯 What it does: Propose the Variational Flow Maps (VFM) framework, which maps noise to the data space by learning a noise sampler conditioned on observations, enabling one-shot conditional generation.
CodeRecommendation SystemAdversarial AttackTransformerPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: A cross-modal interactive poisoning attack called VENOMREC is proposed for multi-modal large language model recommendation systems. It utilizes synchronized visual and textual perturbations to guide the fused representation towards high-exposure semantic directions, achieving targeted promotion of target products.
CodeComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelMixture of ExpertsVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a post-training quantization framework called VEQ for hybrid expert vision-language models, which can simultaneously address modality differences and expert heterogeneity.