ICLR 2026 Papers — Page 54
International Conference on Learning Representations · 5356 papers
Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSs
Yuka Hashimoto (NTT, Inc.), Masahiro Ikeda (University of Osaka)
Explainability and InterpretabilityRepresentation LearningImageTabular
🎯 What it does: This paper proposes an upper bound on Rademacher complexity based on the Koopman operator and RKHS, demonstrating that neural networks with high-rank weight matrices can generalize well.
Why is Your Language Model a Poor Implicit Reward Model?
Noam Razin (Princeton University), Sanjeev Arora (Princeton University)
Explainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigated and compared the generalization performance of explicit reward models (EX-RM) and implicit reward models (IM-RM), proposed theoretical analysis and confirmed through experiments that IM-RM relies more on token-level surface cues, leading to poor generalization.
Why Keep Your Doubts to Yourself? Trading Visual Uncertainties among Vision-Language Models
Jusheng Zhang (Sun Yat-sen University), Keze Wang (Sun Yat-sen University)
Reinforcement LearningAgentic AIMixture of ExpertsVision Language ModelMultimodalityBenchmark
🎯 What it does: This paper proposes the Agora framework, which quantifies the cognitive uncertainty of multimodal vision-language models as tradable assets and achieves multi-agent collaboration through market mechanisms.
Why Less is More (Sometimes): A Theory of Data Curation
Elvis Dohmatob (Concordia University), Reyhane Askari-Hemmat (Meta FAIR)
Data-Centric LearningImageText
🎯 What it does: Propose a high-dimensional theoretical framework that explains under what conditions 'less is more' and provides an exact scaling law;
Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention
Haiquan Qiu (Tsinghua University), Quanming Yao (Tsinghua University)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Analyzes the root cause of loss explosion caused by low-precision Flash Attention in GPT-2 training and provides a minimal modification repair solution.
Why Prototypes Collapse: Diagnosing and Preventing Partial Collapse in Prototypical Self-Supervised Learning
Gabriel Y. Arteaga (University of Oslo), Adín Ramírez Rivera (UiT Arctic University of Norway)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: This paper addresses the partial prototype collapse issue in prototype self-supervised learning by proposing a fully decoupled training strategy. By separating prototype updates from encoder learning and employing an online Gaussian Mixture Model (GMM) for prototype estimation, the approach eliminates collapse and enhances representation diversity.
Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspective
Zhihao Zhang (Fudan University), Kai Chen (Shanghai Artificial Intelligence Laboratory)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningImageTextMultimodalityBenchmark
🎯 What it does: Investigated the knowledge retention effects of SFT and RFT in post-training of multimodal large language models, using jigsaw tasks to detect catastrophic forgetting;
Why We Need New Benchmarks for Local Intrinsic Dimension Estimation
Piotr Tempczyk (IDEAS Research Institute), Adam Kurpisz (BFH Bern Business School)
Data SynthesisConvolutional Neural NetworkFlow-based ModelAuto EncoderImageTabularBenchmark
🎯 What it does: This paper proposes a new benchmark framework for estimating Local Intrinsic Dimension (LID), which can perform consistent evaluation of the same manifold across different domains;
Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs
Feng Hong (Shanghai Jiao Tong University), Jiangchao Yao (Shanghai Jiao Tong University)
Computational EfficiencyLarge Language ModelTextMultimodality
🎯 What it does: Proposed the WINO algorithm, which performs reversible draft-verification parallel decoding on the decoding process of DLLM, significantly improving speed and quality.
WideSearch: Benchmarking Agentic Broad Info-Seeking
Ryan Wong, WANG KE
RetrievalLarge Language ModelAgentic AITextTabularBenchmark
🎯 What it does: Introduces the WideSearch benchmark to evaluate the reliability of large language model-driven search agents in large-scale, comprehensive information retrieval tasks.
Wiki-R1: Incentivizing Multimodal Reasoning for Knowledge-based VQA via Data and Sampling Curriculum
Shan Ning (ShanghaiTech University), Xuming He (ShanghaiTech University)
RetrievalLarge Language ModelReinforcement LearningVision Language ModelMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposes Wiki-R1, a reinforcement learning framework based on controllable data generation and experience curves, to enhance the reasoning capabilities of multimodal large models in knowledge-driven visual question answering.
WILD-Diffusion: A WDRO Inspired Training Method for Diffusion Models under Limited Data
Xianglu Wang, Hu Ding (University Of Science And Technology Of China)
GenerationDiffusion modelImage
🎯 What it does: Proposes the WILD-Diffusion method, which extends the support of diffusion models under limited data through distributionally robust training driven by WDRO, thereby alleviating overfitting and improving generation quality.
WIMFRIS: WIndow Mamba Fusion and Parameter Efficient Tuning for Referring Image Segmentation
Seunghun Moon (Sogang University), Suk-Ju Kang (Sogang University)
SegmentationVision Language ModelContrastive LearningMultimodality
🎯 What it does: Proposed the WIMFRIS framework, combining Window Mamba Fusion (HMF/WMF) and parameter-efficient fine-tuning (MTA, MSA) to achieve efficient and accurate inference for reference image segmentation tasks.
WIMLE: Uncertainty‑Aware World Models with IMLE for Sample‑Efficient Continuous Control
Mehran Aghabozorgi (Simon Fraser University), Ke Li (Simon Fraser University)
Robotic IntelligenceReinforcement LearningWorld ModelBenchmark
🎯 What it does: Propose a multi-modal world model based on IMLE in reinforcement learning, leveraging model uncertainty weighted learning to enhance sample efficiency in continuous control tasks.
WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
Sihan Chen (Renmin University of China), Tianyi Chen (Microsoft)
Computational EfficiencyTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose WINA, a training-free sparse activation method that jointly determines activation using the amplitude of hidden states and the column norm of weights, significantly reducing FLOPs while maintaining LLM inference accuracy.
WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool
Zizun Li (University of Science and Technology of China), Tong He (Shanghai AI Lab)
Pose EstimationDepth EstimationComputational EfficiencyTransformerImagePoint Cloud
🎯 What it does: Propose a real-time online 3D reconstruction framework called WinT3R, which can instantaneously predict camera poses and point cloud maps from continuous image streams.
Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning
Wassim Bouaziz (Meta), El-Mahdi El-Mhamdi (École polytechnique)
Adversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: By inserting a minimal number of indirect poisoning samples into the training data, the LM learns and memorizes a hidden secret sequence during pre-training, enabling dataset ownership verification.
WithAnyone: Toward Controllable and ID Consistent Image Generation
Hengyuan Xu (Fudan University), Yu-Gang Jiang (Fudan University)
GenerationData SynthesisDiffusion modelContrastive LearningImageBenchmark
🎯 What it does: Developed a multi-identity controllable and identity-consistent image generation model called WithAnyone, addressing the copy-paste problem, and released a large-scale multi-identity dataset MultiID-2M and evaluation benchmark MultiID-Bench.
WMPO: World Model-based Policy Optimization for Vision-Language-Action Models
Fangqi Zhu (Hong Kong University of Science and Technology), Song Guo (Hong Kong University of Science and Technology)
Robotic IntelligenceReinforcement LearningVision-Language-Action ModelDiffusion modelWorld ModelVideoTextMultimodality
🎯 What it does: Strategy optimization of vision-language-action (VLA) models based on pixel-level video generation world models, performing on-policy RL entirely in an 'imagined' environment, eliminating the high sampling costs of real robot interactions.
World-In-World: World Models in a Closed-Loop World
Jiahan Zhang (Johns Hopkins University), Jieneng Chen (Johns Hopkins University)
Robotic IntelligenceSupervised Fine-TuningReinforcement LearningVision Language ModelVision-Language-Action ModelWorld ModelImageVideoTextBenchmark
🎯 What it does: Proposes the World-In-World platform, providing a closed-loop evaluation interface and a unified action API, enabling multiple visual world models to perform online planning and decision-making in embedded tasks.
World2Minecraft: Occupancy-Driven Simulated Scenes Construction
Lechao Zhang (East China Normal University), Xin Tan (East China Normal University)
Data SynthesisLarge Language ModelSupervised Fine-TuningVision Language ModelVision-Language-Action ModelImagePoint CloudBenchmarkAudio
🎯 What it does: Propose the World2Minecraft framework, which converts real-world indoor scenes into editable Minecraft environments through 3D semantic occupancy prediction, and construct two large datasets, MinecraftVLN and MinecraftOcc, to verify audio-visual navigation tasks in this environment.
WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark
Wang Lin (Zhejiang University), Alan Yuille (Johns Hopkins University)
GenerationTransformerLarge Language ModelReinforcement LearningVision Language ModelImageMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Developed the WorldEdit dataset and evaluation set, constructing a world knowledge-driven image editing framework targeting implicit causal editing instructions, and implementing two-stage fine-tuning and causal verification rewards on the Bagel model.
WorldGym: World Model as An Environment for Policy Evaluation
Julian Hector Quevedo, Sherry Yang (Stanford University)
Robotic IntelligenceTransformerVision Language ModelDiffusion modelWorld ModelImageVideoText
🎯 What it does: Established a world model based on video generation (WorldGym) for efficient and reproducible evaluation of robot control strategies;
WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs
Jack Hong (Xiaohongshu Inc), Weidi Xie (Xiaohongshu Inc)
TransformerLarge Language ModelVision Language ModelVideoMultimodalityBenchmarkAudio
🎯 What it does: Propose the WorldSense benchmark for multimodal (visual + audio) video question answering, covering 8 domains and 67 subcategories, with 1,662 videos and 3,172 questions.
WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Driving
Ziyue Zhu (Nankai University), jian Yang
Autonomous DrivingDiffusion modelGaussian SplattingVideo
🎯 What it does: Propose the WorldSplat framework to achieve one-time generation of consistent 4D driving scene videos that can be transformed along trajectories.
WorldTree: Towards 4D Dynamic Worlds from Monocular Video using Tree-Chains
Qisen Wang (Beihang University), Jia Li (Beihang University)
GenerationGaussian SplattingOptical FlowVideo
🎯 What it does: Propose the WorldTree framework to achieve 4D dynamic reconstruction from monocular videos, incorporating the Temporal Partition Tree (TPT) and Spatial Ancestor Chain (SAC) to enable coarse-to-fine temporal optimization and multi-level spatial representation;
WOW-Seg: A Word-free Open World Segmentation Model
Danyang Li (Nankai University), Xiang Li (NKIARI)
SegmentationTransformerLarge Language ModelImageBenchmark
🎯 What it does: Proposed a lexical-free open-world segmentation model called WOW-Seg, which automatically identifies arbitrary objects through visual prompts and outputs masks and categories.
WRING Out The Bias: A Rotation-Based Alternative To Projection Debiasing
Walter Gerych (Worcester Polytechnic Institute), Marzyeh Ghassemi (Massachusetts Institute of Technology)
Explainability and InterpretabilityRepresentation LearningImage
🎯 What it does: Proposed a bias elimination method called WRING for vision-language models, which removes known pseudo-correlation bias by rotating embeddings within a subspace instead of projecting.
WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
Changxin Tian (Ant Group), JUN ZHOU
OptimizationTransformerLarge Language ModelMixture of ExpertsTextBenchmark
🎯 What it does: Proposed a non-decaying learning rate scheduling framework WSM (Warmup-Stable and Merge), achieving the learning rate decay effect of pre-training through checkpoint merging.
WSVD: Weighted Low-Rank Approximation for Fast and Efficient Execution of Low-Precision Vision-Language Models
Haiyu Wang (New York University), Sai Qian Zhang (New York University)
CompressionComputational EfficiencyVision Language ModelMultimodalityBenchmark
🎯 What it does: This work proposes the WSVD (Weighted Low-Rank Approximation) framework to compress and accelerate the inference of vision-language models (VLMs) through techniques such as weighted low-rank approximation, head-wise SVD, and quantization-aware fine-tuning, particularly achieving efficient decoding at low precision.
X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model
Jinliang Zheng (Tsinghua University), Xianyuan Zhan (Tsinghua University)
Robotic IntelligenceTransformerPrompt EngineeringVision-Language-Action ModelFlow-based ModelMultimodality
🎯 What it does: Proposed a general Vision-Language-Action model X-VLA, which integrates hardware configurations into the Transformer encoder via soft prompts, achieving unified training and deployment across multiple robot platforms.
XIL: Cross-Expanding Incremental Learning
Heayoun Choi (Chung-Ang University), Eunwoo Kim (Chung-Ang University)
ClassificationDomain AdaptationTransformerPrompt EngineeringDiffusion modelImage
🎯 What it does: Propose a Cross-Domain Incremental Learning (XIL) framework, and implement bidirectional domain knowledge transfer through the XEED method based on domain-specialized prompts, generated samples, and evolved prototypes.
xLSTM Scaling Laws: Competitive Performance with Linear Time-Complexity
Maximilian Beck (Johannes Kepler University), Sepp Hochreiter (Johannes Kepler University)
Computational EfficiencyRecurrent Neural NetworkTransformerText
🎯 What it does: Compare the scaling behavior of xLSTM and Transformer during training and inference, plot the compute-loss Pareto frontier, compute the optimal model and its dependence on context length, and model inference time.
XModBench: Benchmarking Cross-Modal Capabilities and Consistency in Omni-Language Models
Xingrui Wang, Zicheng Liu
TransformerLarge Language ModelPrompt EngineeringImageTextMultimodalityBenchmarkAudio
🎯 What it does: Proposed XModBench, a specialized benchmark for evaluating cross-modal consistency in multimodal large language models, covering six modality combinations of audio, visual, and text;
XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning
Daniel Palenicek (Technical University of Darmstadt), Jan Peters (Technical University of Darmstadt)
OptimizationReinforcement LearningImage
🎯 What it does: Develop and evaluate a deep actor-critic algorithm XQC based on SAC, leveraging batch normalization, weight normalization, and distributed cross-entropy loss to improve the optimization landscape of the Critic network, thereby enhancing sample efficiency.
xRFM: Accurate, scalable, and interpretable feature learning models for tabular data
Daniel Beaglehole (UC San Diego), Mikhail Belkin (UC San Diego)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningHyperparameter SearchTabular
🎯 What it does: Proposed and implemented xRFM, a model that combines kernel regression with recursive feature machine (RFM) and binary tree partitioning for efficient and interpretable tabular data prediction.
YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting
Botao Ye (ETH Zurich), Marc Pollefeys (ETH Zurich)
Pose EstimationTransformerGaussian SplattingImage
🎯 What it does: This paper proposes YoNoSplat, a feedforward 3D Gaussian Splatting model that can rapidly reconstruct high-quality 3D scenes from multi-view images with arbitrary numbers of uncalibrated or calibrated, known or unknown camera poses.
You Point, I Learn: Online Adaptation of Interactive Segmentation Models for Handling Distribution Shifts in Medical Imaging
Wentian Xu (University of Oxford), Konstantinos Kamnitsas (University of Oxford)
SegmentationDomain AdaptationConvolutional Neural NetworkBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This study proposes an online adaptation interactive medical image segmentation framework (OAIMS), which enhances segmentation performance under distribution shifts by adaptively updating the model on a per-image and per-click basis using user click feedback.
Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents
Shuai Shao (Shanghai Artificial Intelligence Laboratory), Jing Shao (Shanghai Jiao Tong University)
Safty and PrivacyTransformerLarge Language ModelAgentic AIPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Studied a new security risk in self-evolving large language model agents called Misevolution, systematically evaluating its performance across four evolutionary paths: model, memory, tools, and workflow.
Your Language Model Secretly Contains Personality Subnetworks
Ruimeng Ye (University of Tulsa), Bo Hui (University of Tulsa)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Extract sparse subnetworks specific to different personas from pre-trained LLMs using activation-guided structured pruning without training steps.
Your Models Have Thought Enough: Training Large Reasoning Models to Stop Overthinking
Jinyi Han (East China Normal University), Yanghua Xiao (East China Normal University)
Computational EfficiencyLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: Training large reasoning models to actively stop unnecessary thinking, thereby enhancing reasoning efficiency
Your VAR Model is Secretly an Efficient and Explainable Generative Classifier
Yi-Chung Chen (Purdue University), Jing Gao (Purdue University)
ClassificationExplainability and InterpretabilityComputational EfficiencyDiffusion modelAuto EncoderContrastive LearningImage
🎯 What it does: This paper proposes a generative classifier based on the visual autoregressive (VAR) model and designs the A-VARC+ method, improving inference efficiency and classification accuracy while maintaining a tractable likelihood.
Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning
Junnan Dong, Xing Sun
RetrievalGraph Neural NetworkLarge Language ModelAgentic AIPrompt EngineeringGraphRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose Youtu-GraphRAG, a vertically unified agentic framework that fully integrates graph construction and retrieval, using graph schema to constrain knowledge extraction, community detection, and retrieval, supporting multi-level reasoning.
YuE: Scaling Open Foundation Models for Long-Form Music Generation
Ruibin Yuan (Hong Kong University of Science and Technology), Yike Guo (Hong Kong University of Science and Technology)
GenerationTransformerLarge Language ModelAuto EncoderTextAudio
🎯 What it does: Developed and open-sourced the open-source music generation foundation model YuE, specifically designed for generating songs from lyrics up to five minutes long, capable of automatically generating complete songs while maintaining alignment between lyrics and melody, structural coherence, and coordination between vocals and accompaniment.
Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning
Ang Li (Columbia University), Micah Goldblum (Columbia University)
Data SynthesisTransformerSupervised Fine-TuningVision Language ModelMultimodalityChain-of-Thought
🎯 What it does: This paper constructs the ZEBRA-COT dataset, containing 182,384 interactive text-image reasoning chains spanning 18 domains and over 50 tasks, and fine-tunes Anole-7B and Bagel-7B on this dataset, significantly enhancing visual chain-of-thought reasoning capabilities;
Zephyrus: An Agentic Framework for Weather Science
Sumanth Varambally (University of California San Diego), Rose Yu (University of California San Diego)
Large Language ModelAgentic AITabularTime SeriesPhysics Related
🎯 What it does: Proposes the ZEPHYRUS framework, integrating LLMs with weather data, forecasts, simulations, climate statistics, and other tools to enable interactive reasoning in weather science.
Zero-Overhead Introspection for Adaptive Test-Time Compute
Rohin Manvi (UC Berkeley), Sergey Levine (UC Berkeley)
Computational EfficiencyTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: Propose a zero-overhead introspective framework ZIP-RC, which leverages logits retained by language models to predict the joint distribution of rewards and computational costs during a single forward pass, enabling real-time self-assessment and dynamic resource allocation during inference.
Zero-Sacrifice Persistent-Robustness Adversarial Defense for Pre-Trained Encoders
Zhuxin Lei (Sichuan University), Yi Zhang (Sichuan University)
Representation LearningAdversarial AttackSupervised Fine-TuningContrastive LearningImage
🎯 What it does: Designed and implemented a dual-branch zero-sacrifice persistent robustness adversarial defense framework called ZePAD, aimed at enhancing the robustness of publicly pre-trained encoders against downstream irrelevant adversarial samples while maintaining or even improving the performance on normal samples.
Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamics
Maksim Bobrin (Computational Imaging Lab), Dmitry V. Dylov
Domain AdaptationTransformerReinforcement LearningWorld Model
🎯 What it does: Investigate the adaptability of BFM under zero-shot conditions for dynamic changes, proposing two methods: Belief-FB and Rotation-FB. By integrating Transformer-based belief estimation and clustering of latent directions within the Forward-Backward (FB) framework, achieve zero-shot adaptation to unseen dynamics.
Zero-shot Forecasting by Simulation Alone
Boris N. Oreshkin (Amazon Science), Andrew Gordon Wilson
Data SynthesisComputational EfficiencyTime Series
🎯 What it does: Developed SarSim0, a fast univariate time series simulation pipeline for zero-shot prediction;
Zero-shot HOI Detection with MLLM-based Detector-agnostic Interaction Recognition
Shiyu Xuan (Nanjing University of Science and Technology), Jinhui Tang (Nanjing University of Science and Technology)
RecognitionObject DetectionTransformerLarge Language ModelPrompt EngineeringImageTextMultimodality
🎯 What it does: Propose a decoupled framework that separates object detection from interaction recognition in zero-shot human-object interaction detection, leveraging multimodal large language models (MLLMs) to convert interaction recognition into a deterministic visual question-answering task, achieving training-free interaction recognition, and enhancing performance and efficiency through spatial-aware pooling and single deterministic matching.
Zero-shot Human Pose Estimation using Diffusion-based Inverse solvers
Sahil Bhandary Karnoor (University of Illinois at Urbana-Champaign), Romit Roy Choudhury (University of Illinois at Urbana-Champaign)
Pose EstimationDiffusion model
🎯 What it does: Proposes the InPose method, which achieves zero-shot full-body pose estimation using a diffusion-based inverter from rotation and position measurements of only three-point sensors.
ZeroGR: A Generalizable and Scalable Framework for Zero-Shot Generative Retrieval
Weiwei Sun (Carnegie Mellon University), Yiming Yang (Carnegie Mellon University)
RetrievalTransformerLarge Language ModelText
🎯 What it does: Developed a generative retrieval framework called ZEROGR that can be used in zero-shot scenarios and applied to diverse document retrieval tasks.
Zeros can be Informative: Masked Binary U-Net for Image Segmentation on Tensor Cores
Chunshu Wu (Pacific Northwest National Laboratory), Ang Li (Pacific Northwest National Laboratory)
SegmentationComputational EfficiencyConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: Propose Masked Binary U-Net and achieve efficient inference on Tensor Core, addressing computational and energy efficiency bottlenecks for real-time segmentation on edge devices.
ZeroSiam: An Efficient Asymmetry for Test-Time Entropy Optimization without Collapse
Guohao Chen (Nanyang Technological University), Zhiqi Shen (Nanyang Technological University)
Domain AdaptationOptimizationConvolutional Neural NetworkTransformerLarge Language ModelImageTextBenchmark
🎯 What it does: Propose ZeroSiam, an asymmetric Siamese architecture that minimizes entropy during testing to prevent models from collapsing into degenerate solutions caused by one-hot encoding;
ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Training
Feijiang Han (University of Pennsylvania), Lyle Ungar (University of Pennsylvania)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose ZeroTuning, a training-free method that significantly improves model performance by only adjusting the attention weights of the initial token (e.g., <BOS>) during inference;