Conference on Neural Information Processing Systems · 2283 papers
Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens
Xixian Yong (Renmin University of China), Xian Wu (Tencent)
CodeComputational EfficiencyTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Quantify the thinking efficiency of large reasoning models from an information-theoretic perspective and propose an entropy-based adaptive stopping strategy to reduce the length of reasoning chains.
Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models
Jiaqi WANG, Mike Zheng Shou (National University of Singapore)
CodeTransformerSupervised Fine-TuningReinforcement LearningVision Language ModelMultimodality
🎯 What it does: A two-stage training framework called TON is designed to teach visual-language models when to reason in reinforcement learning, significantly reducing reasoning length while maintaining or even improving performance.
CodeCompressionTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: A framework called CoLaR is proposed, which can dynamically compress the LLM inference chain in the latent space, supporting 'silent' inference and allowing for dynamic adjustment of inference speed through a compression factor.
Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models
Ilgee Hong (Georgia Institute of Technology), Tuo Zhao (Amazon)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
🎯 What it does: A generative reward model framework named Think-RM was designed and trained, capable of generating reasoning chains of up to thousands of tokens through an internal 'thinking' process, and a training pipeline for RLHF using pairwise preference was proposed;
Stephen Chung (University of Cambridge), Jie Fu (Shanghai AI Lab)
CodeTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: The Thinker task is proposed, which divides single-turn QA into four stages (quick thinking, verification, slow thinking, and summarization), training the LLM's intuition, evaluation, refinement, and integration abilities through multi-stage reward mechanisms.
Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoning
Yihong Tang (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeKnowledge DistillationLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: A role-aware reasoning (RAR) method is proposed, enabling large language models to generate internal thoughts that align with character settings, addressing issues of character deviation and style drift.
CodeRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningTextMultimodality
🎯 What it does: This paper proposes a novel testing moment scale dimension called Interaction Scaling, which enhances the agent's information acquisition and behavior adjustment capabilities in dynamic environments by increasing the number of interaction steps. Based on this, we designed TTI (Test-Time Interaction) — an online reinforcement learning framework that adapts the agent to extend the interaction length during deployment.
This Time is Different: An Observability Perspective on Time Series Foundation Models
Ben Cohen (Datadog AI Research), Othmane Abou-Amal (Datadog AI Research)
CodeTransformerTime SeriesBenchmark
🎯 What it does: A zero-copy pre-training model TOTO specifically designed for observable time series is proposed, and a large observable data benchmark BOOM is constructed.
Thompson Sampling in Function Spaces via Neural Operators
Rafael Oliveira (CSIRO Data61), Edwin V. Bonilla (CSIRO Data61)
CodeOptimizationReinforcement LearningTabular
🎯 What it does: A Thompson Sampling method based on Neural Operators is proposed for optimizing known functional objectives of unknown operators in function space.
Thoughts Are All Over the Place: On the Underthinking of Long Reasoning Models
Yue Wang (Tencent), Dong Yu (Tencent)
CodeTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: This study investigates the phenomenon of 'underthinking' in long reasoning models (LRMs), proposes a token efficiency-based metric, and designs a Thinking Interruption Penalty (TIP) decoding strategy to suppress premature switching of thought processes, thereby enhancing reasoning efficiency and accuracy.
Tightening Regret Lower and Upper Bounds in Restless Rising Bandits
Cristiano Migali (Politecnico di Milano), Alberto Maria Metelli (Politecnico di Milano)
CodeOptimizationReinforcement Learning from Human FeedbackTabular
🎯 What it does: This paper theoretically studies the Rising and Rising Concave multi-armed bandit (MAB) problems, providing lower and upper bounds for both types of problems, and proposes a new algorithm called RC-BE.
🎯 What it does: Designed and implemented the Tiled Flash Linear Attention (TFLA) algorithm and its efficient kernel on mLSTM, and proposed a Sigmoid input gate version of mLSTM to enhance the computational efficiency and training stability of long sequence linear RNNs.
🎯 What it does: This paper introduces time-reversal symmetry and designs a TR-DRL framework to enhance the sample efficiency of deep reinforcement learning in robotic manipulation tasks.
Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach
Tal Gonen (Ben-Gurion University of Negev), Omri Azencot (Ben-Gurion University of Negev)
CodeGenerationData SynthesisDiffusion modelTime SeriesSequentialFinance Related
🎯 What it does: A unified time series generation framework is proposed and implemented, combining cross-domain pre-training and few-shot fine-tuning, capable of generating high-quality time series with extremely low data volumes.
🎯 What it does: A temporal latent variable model, TE-ViDS, is proposed to learn low-dimensional temporal evolution representations from neural firing in the visual cortex of mice, further decomposed into stimulus-related external latent variables and internal latent variables influenced by internal states.
TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting
Jaebin Lee (Sungkyunkwan University), Hankook Lee (Sungkyunkwan University)
CodeTransformerTime Series
🎯 What it does: The TIMEPERCEIVER framework is proposed, unifying the encoder, decoder, and training strategy, enabling multivariate time series forecasting for extrapolation, interpolation, and filling of time segments at any position.
TimeWak: Temporal Chained-Hashing Watermark for Time Series Data
Zhi Wen Soi (University of Neuchâtel), Lydia Y. Chen (Delft University of Technology)
CodeDiffusion modelTime SeriesFinance Related
🎯 What it does: This paper proposes TimeWak, a generative watermarking scheme for multivariate time series diffusion models that can embed detectable watermarks in the data space.
TITAN: A Trajectory-Informed Technique for Adaptive Parameter Freezing in Large-Scale VQE
Yifeng Peng (Stevens Institute of Technology), Yuxuan Du (Nanyang Technological University)
CodeOptimizationConvolutional Neural NetworkReinforcement LearningTabularPhysics Related
🎯 What it does: The TITAN framework is proposed and implemented, utilizing deep learning to predict and freeze redundant parameters in VQE in advance, thereby reducing measurement overhead while maintaining or improving energy estimation accuracy.
To Think or Not To Think: A Study of Thinking in Rule-Based Visual Reinforcement Fine-Tuning
Ming Li (Shanghai AI Laboratory), Kaipeng Zhang (Shanghai AI Laboratory)
CodeClassificationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringImageMultimodality
🎯 What it does: This study investigates the impact of explicit thinking processes in Rule-based Fine-Tuning (RFT) on Multimodal Large Language Models (MLLMs), proposing methods such as No-Thinking RFT, Think-After-Answer, and Adaptive-Thinking, and conducting systematic experiments on tasks like image classification and visual reasoning.
🎯 What it does: A training-independent and condition-independent Token Perturbation Guidance (TPG) method is proposed to enhance the generation quality and semantic alignment of diffusion models.
TokenSwap: A Lightweight Method to Disrupt Memorized Sequences in LLMs
Parjanya Prajakta Prashant (University of California San Diego), Babak Salimi (University of California San Diego)
CodeGenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a method to prevent large language models from producing memorized outputs during the post-inference stage—TokenSwap. It suppresses the model's direct reproduction of training data by replacing probabilities on high-frequency grammatical vocabulary with those from a small auxiliary model.
🎯 What it does: A framework named TOMCAT is proposed, which continuously accumulates multimodal knowledge (visual and textual) using unlabeled data during the testing phase of CZSL, and updates category prototypes through adaptive weighting to address the issue of label space distribution drift.
Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task
Sunqi Fan (Tsinghua University), Shuojin Yang (Tsinghua University)
CodeRecognitionObject DetectionComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringVideo
🎯 What it does: This paper constructs a lightweight video toolbox consisting of 22 types that cover space, time, and general functions, and proposes a Star Alternating Time-Space Reasoning framework (STAR) to achieve step-by-step localization and reasoning of 3D RoI in video question-answering tasks.
Cheng Qian (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
CodeReinforcement Learning
🎯 What it does: This paper systematically studies reward design in reinforcement learning and proposes a refined reward framework for Tool Integrated Reasoning (TIR). It implements the ability to use tools in LLMs from scratch using various RL algorithms (GRPO, PPO), significantly improving tool selection and invocation performance.
CodeGenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes and implements an entropy-constrained adaptive sampling method called Top-H, aimed at balancing creativity and coherence in large language model generation.
🎯 What it does: A low-dimensional graph embedding method called TopER based on topological data analysis is proposed, which uses the filtering sequence of the graph to obtain two coefficients (intercept and slope) through linear regression to represent the structural evolution of the graph.
🎯 What it does: By clustering the internal hidden states of large reasoning models to construct reasoning graphs, we systematically analyze their loops, diameters, and small-world characteristics, exploring the relationship between these graph structures and reasoning performance.
🎯 What it does: An adaptive consistency prediction framework STACI aimed at flow networks is designed, which implements multi-site joint uncertainty quantification using topology-aware inconsistency scores and adaptive confidence levels.
🎯 What it does: A continuous tubular manifold representation based on spherical B-spline curves (BBSC) is proposed, and an SE(3)-BBSCformerGCN framework is constructed by combining SE(3)-equivariant networks with graph convolutional networks for learning geometric and topological features of tubular structures.
🎯 What it does: This paper proposes a new framework called TopoPoint, which explicitly detects lane endpoints and jointly infers lane information, significantly improving the accuracy of road topology inference.
🎯 What it does: By accelerating diffusion model inference through multi-rate integration, a training-independent Tortoise and Hare Guidance (THG) method is proposed, significantly reducing the number of function evaluations while maintaining generation quality.
Toward Interpretable Evaluation Measures for Time Series Segmentation
Félix Chavelli, Michaël Thomazo
CodeSegmentationExplainability and InterpretabilityTime Series
🎯 What it does: An interpretable time series segmentation evaluation method is proposed, addressing the shortcomings of traditional metrics that fail to capture the location and type of errors, with the design of two new metrics, WARI and SMS.
Toward Relative Positional Encoding in Spiking Transformers
Changze Lv (Fudan University), Dongsheng Li (Microsoft Research Asia)
CodeSpiking Neural NetworkTransformerImageTextTime Series
🎯 What it does: Two relative position encoding methods, Gray-PE and Log-PE, are proposed and applied to the spiking Transformer, improving the self-attention mechanism to XNOR logic.
🎯 What it does: The GyroAtt framework is proposed, which extends the self-attention mechanism to the general gyrovector space, achieving a unified treatment of various matrix manifolds such as SPD, SPSD, and Grassmannian.
Towards a Golden Classifier-Free Guidance Path via Foresight Fixed Point Iterations
Kaibo Wang (Hong Kong University of Science and Technology), Yang Xiang (Hong Kong University of Science and Technology)
CodeGenerationOptimizationImage
🎯 What it does: This paper views conditional guidance as a calibration process for the 'golden path' and provides a unified explanation of CFG and its variants through fixed-point iteration.
🎯 What it does: WhAM is proposed, a transformer-based model capable of generating and converting any audio into the audio style of whale coda, and providing classification functionality for coda.
Towards Accurate Time Series Forecasting via Implicit Decoding
Xinyu Li (University of Melbourne), Mingming Gong (University of Melbourne)
CodeRecurrent Neural NetworkTransformerTime Series
🎯 What it does: Improved the decoding phase of time series forecasting by proposing the Implicit Forecaster module, which implicitly predicts future sequences using frequency waveforms.
Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy
Xiaoxiao Ma (University of Science and Technology of China), Lin Ma (Meituan)
CodeGenerationComputational EfficiencyImage
🎯 What it does: This study investigates the sampling problem in autoregressive image generation and proposes an entropy-based dynamic temperature control and entropy-aware inference acceleration strategy.
Alejandro Hernández-Cano (École Polytechnique Fédérale de Lausanne), Martin Jaggi (École Polytechnique Fédérale de Lausanne)
CodeTransformerLarge Language ModelText
🎯 What it does: Proposed and implemented the FOG architecture, achieving complete FP8 GEMM training within the Transformer block, including the attention mechanism, significantly enhancing the throughput of large-scale LLM training.
Towards General Continuous Memory for Vision-Language Models
Wenyi WU, Biwei Huang (University of California)
CodeRetrievalCompressionTransformerVision Language ModelMultimodalityRetrieval-Augmented Generation
🎯 What it does: A pluggable continuous memory module CoMEM is designed for visual-language models (VLM), using the VLM itself as a memory encoder, and utilizing a small amount of self-synthesized data with LoRA fine-tuning, only increasing parameters by 1.2%.
🎯 What it does: This paper proposes viewing the detection of generated images as OOD detection and designs a training-free detection framework called DEnD based on the differential energy of self-supervised models.
Towards Identifiability of Hierarchical Temporal Causal Representation Learning
Zijian Li (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
CodeGenerationRepresentation LearningFlow-based ModelAuto EncoderTime SeriesMagnetic Resonance ImagingFinance Related
🎯 What it does: This study investigates the identifiability of hierarchical latent variables in time series and proposes a recognition framework and generative model based on Causal Hierarchical Latent Dynamics (CHiLD).
🎯 What it does: A black-box pixel-level attack method for scene text recognition models is proposed—Multi-Population Co-evolutionary Search (MPCS), which can cause the model to predict more incorrect characters while maintaining visual semantic integrity.
Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integration
Wenju Sun (Beijing Jiaotong University), Boyang Li (Nanyang Technological University)
CodeOptimizationTransformerImageMultimodality
🎯 What it does: This paper proposes the Layer-wise Optimal Task Vector Merging (LOT Merging) method, which achieves multi-task model merging by minimizing feature drift at the layer level.
🎯 What it does: The KIDOT framework is proposed, modeling medical image reconstruction as a dynamic optimal transport process constrained by imaging physics, and learning to reconstruct from unpaired data through neural networks;
🎯 What it does: The AEOS-Bench benchmark and AEOS-Former scheduling model are proposed for real AEOS constellation scheduling problems, supporting large-scale, dynamic, and constrained scenarios.
Towards Reliable LLM-based Robots Planning via Combined Uncertainty Estimation
Shiyuan Yin (Henan University of Technology), Xuelong Li (China Telecom)
CodeKnowledge DistillationRobotic IntelligenceTransformerLarge Language ModelText
🎯 What it does: The CURE framework is proposed to provide fine-grained uncertainty estimation for robot planning generated by large language models (LLMs) to enhance the reliability of planning.
🎯 What it does: This paper proposes a robust parameter-efficient fine-tuning framework (RFedLR) specifically designed for scenarios with label noise in federated learning, combining sensitivity-aware robust tuning (SRT) and adaptive LoRA aggregation (AFLA);
🎯 What it does: In response to pseudo-label learning in semantic segmentation, the authors propose using Error-Correcting Output Codes (ECOC) for fine-grained multi-bit binary encoding of categories, thereby enhancing the robustness of pseudo-labels and improving model training.
🎯 What it does: A new zero-shot RL method based on the Forward-Backward (FB) framework, called BREEZE, is proposed to address issues such as scale inconsistency, bias, and outlier estimation in FB methods.
🎯 What it does: The Cauvis method is proposed, which achieves reverse causal adjustment in single-source domain generalization object detection through visual prompts and cross-attention, and incorporates a dual-branch adapter to decouple causal features from high-frequency domain features.
🎯 What it does: This paper proposes MU-SplitFed, a method to alleviate stragglers in Split Federated Learning through server-side unbalanced updates and zeroth-order optimization, significantly reducing the number of communication rounds and decoupling training time from the slowest client.
🎯 What it does: This paper proposes a new framework called SynDR-IQA, which enhances the cross-domain generalization ability of no-reference image quality assessment (BIQA) models by reshaping the distribution of synthetic data.
🎯 What it does: This paper proposes a network fingerprint embedding method based on the frequency components of convolution kernels, theoretically proving that these components remain unchanged during the fine-tuning process, thereby achieving robust protection of model copyrights.
Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
Wenkai Yang (Renmin University of China), Furu Wei (Microsoft Research)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkChain-of-Thought
🎯 What it does: This paper studies the test-time scalability of chain-of-thought (CoT) length in reasoning tasks using large language models, finding that overly long CoTs can lead to a decline in reasoning performance. It then proposes the Thought Optimal Expansion (TOPS) strategy, allowing the model to determine the necessary CoT length on its own and achieve more efficient and effective System 2 reasoning through self-improvement. Ultimately, it achieves better performance than existing distilled o1 models on multiple mathematical reasoning benchmarks.
Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons
Jianhui Chen (Tsinghua University), Juanzi Li (Tsinghua University)
CodeSafty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Through mechanism interpretability methods, identify and validate safety neurons in large language models, revealing the competitive relationship between safety and usefulness, and construct safety protections based on these neurons to preemptively detect harmful outputs.
🎯 What it does: This paper proposes the UAE-3D multimodal VAE and UDM-3D joint latent diffusion model, which compresses the originally separate invariant and equivariant modalities into the same latent space, achieving efficient 3D molecular generation.
Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization
Ming Nie (Fudan University), Li Zhang (Fudan University)
CodeGenerationOptimizationReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: Through the warm-up phase and training based on GRPO reinforcement learning, the unified visual language model is able to perform multimodal interactive generation (text and images appearing alternately) in a high-quality and coherent manner.
🎯 What it does: This paper proposes DiDA, an unsupervised domain bridging framework based on image degradation, aimed at enhancing the cross-domain generalization ability of semantic segmentation.
Towards Unsupervised Open-Set Graph Domain Adaptation via Dual Reprogramming
Zhen Zhang (Nanjing University), Bingsheng He (National University of Singapore)
CodeDomain AdaptationGraph Neural NetworkGraph
🎯 What it does: This study investigates the problem of unsupervised open set graph domain adaptation, where the target graph contains new categories that do not exist in the source graph.
🎯 What it does: An unsupervised training framework for Graph Edit Distance (GED) called GEDRanker is proposed, which utilizes a GAN discriminator to guide the matching model in generating high-quality node matching matrices and can recover the edit paths.
CodeAdversarial AttackLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: An automated jailbreak framework named Visualization-of-Thought Attack (VoTA) is proposed, which induces VLM to generate unsafe content through image sequences.
ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training
Xin Yao (Central South University), Ming Zhao (Central South University)
CodeAdversarial AttackTransformerVision Language ModelImageText
🎯 What it does: Proposes the ToxicTextCLIP framework, which achieves control over the model during the CLIP pre-training phase through text poisoning and backdoor attacks, mainly including a background-aware text selector and a background-driven text enhancer.
TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding
Shukai Gong (Renmin University of China), Feng Zhou (Renmin University of China)
CodeGenerationComputational EfficiencyTransformerLarge Language ModelTime SeriesSequential
🎯 What it does: The TPP-SD method is proposed, which accelerates event generation by introducing speculative decoding in the sampling of Transformer temporal point processes.
🎯 What it does: TRACE is proposed, a multimodal retriever capable of dual-layer alignment between multivariate time series and corresponding textual descriptions, and it can also serve as a powerful encoder for prediction and classification tasks.
🎯 What it does: A method for white matter fiber tracking called TractoTransformer, which integrates 3D CNN and Transformer, has been developed to reconstruct white matter fiber bundles from diffusion MRI.
Train on Pins and Test on Obstacles for Rectilinear Steiner Minimum Tree
Xingbo Du (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
CodeOptimizationReinforcement LearningGraph
🎯 What it does: A reinforcement learning-based OAREST framework is proposed, capable of generating optimal obstacle-avoiding straight-line Steiner trees without trained obstacles.
🎯 What it does: A continuous learning framework P&M is proposed, which performs model fusion after each task using convex combinations and task vector perturbations to reduce catastrophic forgetting and enhance generalization.
Training Language Models to Generate Quality Code with Program Analysis Feedback
Feng Yao (University of California San Diego), Jingbo Shang (Microsoft Research)
CodeAI Code AssistantReinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes the REAL framework, which utilizes program analysis as a reward signal to train LLMs through reinforcement learning to generate code that is both functionally correct and safe, as well as maintainable.
Daman Arora (Carnegie Mellon University), Andrea Zanette (Carnegie Mellon University)
CodeComputational EfficiencyTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Training large inference models to reduce unnecessary computations during inference mainly by shortening the length of chain-of-thought (CoT) to lower inference costs.
Training Robust Graph Neural Networks by Modeling Noise Dependencies
Yeonjun In (KAIST), Chanyoung Park (KAIST)
CodeGraph Neural NetworkGraph
🎯 What it does: In the study of robustness in graph neural networks, a Dependency Noise (DANG) model is proposed, and DA-GNN is constructed to capture the causal relationships of noise through variational inference.
Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
Haizhou Shi (Rutgers University), Hao Wang (Rutgers University)
CodeTransformerLarge Language ModelText
🎯 What it does: A training-independent Bayesian framework TFB is proposed, which transforms the pre-trained low-rank adapter LoRA into a Bayesian model, allowing for uncertainty estimation without further training.
Training-Free Constrained Generation With Stable Diffusion Models
Stefano Zampini (Polytechnic of Turin), Ferdinando Fioretto (University of Virginia)
CodeGenerationOptimizationDiffusion modelImage
🎯 What it does: A training-free, constraint generation method based on robust diffusion models is proposed, achieving real-time satisfaction of strict constraints such as physical, functional, or copyright constraints through the use of proximal mapping and gradient projection in the latent space.
🎯 What it does: The TreeG framework is proposed, achieving untrained guided generation through tree search, suitable for both continuous and discrete diffusion and flow models, and addressing non-differentiable objectives.
Training-Free Safe Text Embedding Guidance for Text-to-Image Diffusion Models
Byeonghu Na (Korea Advanced Institute of Science and Technology), Il-chul Moon
CodeGenerationSafty and PrivacyDiffusion modelImageText
🎯 What it does: A training-free safe text-to-image diffusion model method is proposed—Safe Text Embedding Guidance (STG), which achieves safe output by dynamically adjusting text embeddings during the sampling process.
Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training
Brian R. Bartoldson (Lawrence Livermore National Laboratory), Bhavya Kailkhura (Lawrence Livermore National Laboratory)
CodeTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes Trajectory Balance with Asynchrony (TBA), a post-training framework that combines the offline Trajectory Balance objective with distributed asynchronous search, decoupling exploration from learning.
Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoning
Yurun Yuan (University of Wisconsin-Madison), Tengyang Xie (University of Wisconsin-Madison)
CodeTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposed and implemented the Trajectory Bellman Residual Minimization (TBRM) algorithm, using the logits of the LLM itself as Q-values, with single-trajectory offline training, eliminating the need for complex components such as critics and importance sampling;
TrajMamba: An Efficient and Semantic-rich Vehicle Trajectory Pre-training Model
Yichen Liu (Beijing Jiaotong University), Huaiyu Wan (Beijing Jiaotong University)
CodeCompressionAutonomous DrivingKnowledge DistillationRepresentation LearningContrastive LearningTime Series
🎯 What it does: This paper proposes an efficient and semantically rich vehicle trajectory pre-training model called TrajMamba, which can simultaneously capture motion patterns and learn travel semantics from both GPS and road perspectives.
🎯 What it does: A cross-market transfer dynamic pricing framework CM-TDP is proposed, compatible with offline to online and online to online transfers, applicable to linear and RKHS nonlinear utility models.
Transferring Linear Features Across Language Models With Model Stitching
Alan Chen (Brown University), Ellie Pavlick (Brown University)
CodeLarge Language ModelAuto EncoderText
🎯 What it does: Proposes and validates a method for transferring Sparse Autoencoders (SAE), detectors, and guiding vectors between language models of different scales using linear mapping (model stitching);
TransferTraj: A Vehicle Trajectory Learning Model for Region and Task Transferability
Tonglong Wei (Beijing Jiaotong University), Huaiyu Wan (Beijing Jiaotong University)
CodeGenerationAutonomous DrivingTransformerMixture of ExpertsTime Series
🎯 What it does: This paper proposes a vehicle trajectory learning model called TransferTraj, which can transfer across different regions and tasks, addressing the issue that traditional models require separate training for each region and task.
🎯 What it does: A brain encoding model based on the Transformer attention mechanism is proposed, dynamically routing retinal spatial features to higher-order visual areas to predict fMRI signals during natural scene viewing.
Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning
Jiaru Zou (University of Illinois Urbana-Champaign), Jingrui He (University of Illinois Urbana-Champaign)
CodeRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes the Transformer Copilot framework, which records and utilizes the model's own error logs (Mistake Log) during the fine-tuning process of LLMs to enhance inference performance.
Felix Draxler (University of California), Stephan Mandt (Chan Zuckerberg Initiative)
CodeTransformerFlow-based ModelTime SeriesBiomedical DataElectronic Health Records
🎯 What it does: A unified Transformer-based intensity-free point process (FLEXTPP) is proposed, capable of handling variable-length event sequences with both discrete and continuous labels, and achieving structured prediction through conditional input.
TransMLA: Migrating GQA Models to MLA with Full DeepSeek Compatibility and Speedup
Fanxu Meng (Institute for Artificial Intelligence Peking University), Muhan Zhang (Institute for Artificial Intelligence Peking University)
CodeCompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Transfer the existing GQA-based pre-trained large models (such as LLaMA, Qwen, etc.) to the MLA structure, and achieve significant inference acceleration by compressing the KV cache while maintaining or only slightly losing performance.
Transstratal Adversarial Attack: Compromising Multi-Layered Defenses in Text-to-Image Models
Chunlong Xie (Chongqing University), Tao Xiang (Chongqing University)
CodeGenerationAdversarial AttackLarge Language ModelImageText
🎯 What it does: A black-box attack framework based on LLM-generated candidate words and genetic optimization is proposed, capable of simultaneously breaking through the multi-layer security defenses of text-to-image models, generating implicit NSFW prompts while evading image filters while maintaining subjective undesirability.