ICLR 2026 Papers — Page 22
International Conference on Learning Representations · 5356 papers
Hierarchical Concept-based Interpretable Models
Oscar Hill (University of Cambridge), Mateja Jamnik (University of Cambridge)
Explainability and InterpretabilityRepresentation LearningVision Language ModelAuto EncoderImage
🎯 What it does: This paper proposes Concept Splitting to automatically discover sub-concepts from CEM embeddings and designs the HiCEM architecture for hierarchical concept modeling, thereby providing fine-grained explainability and intervenability using only coarse-grained concept annotations.
Hierarchical Encoding Tree with Modality Mixup for Cross-modal Hashing
Zhiping Xiao (University of Washington), Ming Zhang (Peking University)
RetrievalMultimodality
🎯 What it does: Propose the HINT method, which constructs a cross-modal encoding tree using hierarchical structure entropy, generates proxy samples, and adopts an adaptive MMD-driven mixup with global consistency learning to achieve unsupervised cross-modal hash retrieval.
Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusion
Dan Haramati (Brown University), George Konidaris (Brown University)
Robotic IntelligenceTransformerReinforcement LearningDiffusion modelImageTabularBenchmark
🎯 What it does: Proposed a hierarchical entity-centric framework based on offline goal-conditioned reinforcement learning, utilizing a fact-based subgoal diffuser to generate sparse, reachable subgoals, significantly improving the success rate of long-horizon tasks.
Hierarchical Multi-Scale Molecular Conformer Generation
Jiapeng Hu (North Carolina State University), Xiaorui Liu (North Carolina State University)
Drug DiscoveryGraph Neural NetworkDiffusion modelBiomedical Data
🎯 What it does: Propose a hierarchical multi-scale molecular conformation generation framework named MSGEN, which first generates heavy atom skeletons as global guidance and then progressively refines hydrogen atoms or finer-grained substructures;
Hierarchical Multi-Stage Recovery Framework for Kronecker Compressed Sensing
Yanbin He (Delft University of Technology), Geethu Joseph (Delft University of Technology)
OptimizationComputational Efficiency
🎯 What it does: Proposes a hierarchical multi-stage recovery framework (MSR) for Kronecker structure compressed sensing (KCS), capable of simultaneously handling three sparsity models: standard sparsity, hierarchical sparsity, and Kronecker support sparsity;
Hierarchical Prototype Learning for Semantic Segmentation
Seoha Lim (Yonsei University), Sung-Bae Cho (Yonsei University)
SegmentationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: Designed a hierarchical prototype learning-based semantic segmentation method called HiPoSeg, which constructs a multi-level prototype space using hierarchical alignment and contrastive learning to achieve a coarse-to-fine recognition process.
Hierarchical Semantic-Acoustic Modeling via Semi-Discrete Residual Representations for Expressive End-to-End Speech Synthesis
Yixuan Zhou (Tsinghua University), Zhiyuan Liu (Tsinghua University)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelFlow-based ModelTextAudio
🎯 What it does: This study proposes an end-to-end hierarchical semantic-acoustic model, achieving expressible and stable speech synthesis through semi-discrete residual representations.
Hierarchical Value-Decomposed Offline Reinforcement Learning for Whole-Body Control
Zhilong Zhang (Nanjing University), Yang Yu (Nanjing University)
Robotic IntelligenceTransformerReinforcement LearningDiffusion modelMultimodality
🎯 What it does: Studied the use of offline reinforcement learning and hierarchical value decomposition to learn whole-body robot control, achieving efficient learning on a large amount of non-expert demonstration data.
Hierarchy Decoding: A Training-free Parallel Decoding Strategy for Diffusion Large Language Models
Xiaojing Qi (Tsinghua University), Da Zheng (Ant Group)
Computational EfficiencyLarge Language ModelDiffusion modelText
🎯 What it does: Propose a hierarchical recursive decoding framework, Hierarchy-dLLM, for discrete diffusion large language models (dLLMs), which significantly accelerates inference while maintaining or improving generation quality.
Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks
Shuo He (Nanyang Technological University), Bo An (Nanyang Technological University)
OptimizationTransformerLarge Language ModelReinforcement LearningAgentic AIText
🎯 What it does: Propose a new hierarchical grouping + adaptive weighting advantage estimation method (HGPO) to address the advantage estimation bias caused by historical context inconsistency in long-term proxy tasks.
HierLoc: Hyperbolic Entity Embeddings for Hierarchical Visual Geolocation
Hari Krishna Gadi (Huawei), Liqiu Meng (Technical University Of Munich)
ClassificationRetrievalRepresentation LearningContrastive LearningImageMultimodality
🎯 What it does: Transforms the visual geolocation task from image-to-image retrieval to image-to-hierarchical entity alignment using hierarchical entity embeddings in hyperbolic space;
HiFo-Prompt: Prompting with Hindsight and Foresight for LLM-based Automatic Heuristic Design
ChentongChen, Ye Fan (Northwest Polytechnical University)
OptimizationTransformerLarge Language ModelPrompt Engineering
🎯 What it does: Propose HiFo‑Prompt, an automated heuristic design framework for large language models (LLMs), which achieves global control and knowledge accumulation through two modules: Foresight and Hindsight.
High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuning
Tim Franzmeyer (University of Oxford), Madian Khabsa (Meta)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a post-training fine-tuning method called HALT, which enables large language models to generate answers only when internal confidence is high, and partially refrain from answering when uncertain;
High Probability Bounds for Non-Convex Stochastic Optimization with Momentum
Shaojie Li (National University of Singapore), Yong Liu (Renmin University of China)
Optimization
🎯 What it does: This paper studies the high probability convergence and generalization bounds of stochastic gradient descent with momentum (SGDM) in non-convex optimization, providing a hierarchical theoretical analysis from general non-convex to Polyak-Łojasiewicz (PL) and then to those with Bernstein conditions.
High-Dimensional Analysis of Single-Layer Attention for Sparse-Token Classification
Nicholas Barnfield (Harvard University), Yue M. Lu (Harvard University)
ClassificationTransformerSequential
🎯 What it does: This paper proposes and analyzes the advantages of single-layer attention networks over traditional linear classifiers in sparse, weak, and rare signal detection tasks;
High-dimensional Analysis of Synthetic Data Selection
Parham Rezaei (Institute of Science and Technology Austria), Marco Mondelli (Institute of Science and Technology Austria)
Data SynthesisDomain AdaptationConvolutional Neural NetworkTransformerDiffusion modelGenerative Adversarial NetworkContrastive LearningImageTextBiomedical Data
🎯 What it does: This paper studies how synthetic data affects model generalization from both theoretical and experimental perspectives. First, under a high-dimensional linear regression framework, it is derived that only covariance differences influence test error, while mean differences do not. Subsequently, a strategy for aligning covariance matrices is proposed in synthetic data selection, and the effectiveness of this strategy in training deep networks is validated across various real-world scenarios.
High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
Aukosh Jagannath (University of Waterloo), Varnan Sarangian (University of Waterloo)
OptimizationStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Investigate the behavior of online stochastic gradient descent (SGD) in the high-dimensional limit, particularly versions with Polyak momentum (SGD-M) and adaptive step sizes (e.g., gradient normalization), and provide differential dynamics and fixed point analysis.
High-dimensional Mean-Field Games by Particle-based Flow Matching
Jiajia Yu (Duke University), Xiuyuan Cheng (Georgia Institute of Technology)
Image TranslationOptimizationFlow-based ModelAuto EncoderImage
🎯 What it does: Propose a proximal fixed-point algorithm based on particle and flow matching for solving high-dimensional mean-field games
High-Probability Bounds for the Last Iterate of Clipped SGD
Savelii Chezhegov (BRAIn Lab), Eduard Gorbunov (Mohamed bin Zayed University of Artificial Intelligence)
Optimization
🎯 What it does: Studied the problem of minimizing a convex objective under only noisy gradient estimates, and for the first time established a high-probability convergence guarantee for the final iteration of clipped stochastic gradient descent (Clipped-SGD).
Highly Efficient and Effective LLMs with Multi-Boolean Architectures
Ba-Hien Tran (Huawei Paris Research Center), Van Minh Nguyen (Huawei Paris Research Center)
Knowledge DistillationLarge Language ModelTextBenchmark
🎯 What it does: Proposed a multi-Boolean kernel (MBOK) framework that trains and fine-tunes large language models directly in the Boolean domain, significantly reducing weight storage and computational complexity.
HiGS: History-Guided Sampling for Plug-and-Play Enhancement of Diffusion Models
Seyedmorteza Sadat (ETH Zürich), Romann M. Weber (DisneyResearch|Studios)
GenerationComputational EfficiencyDiffusion modelImage
🎯 What it does: Proposed HiGS (History-Guided Sampling), which leverages historical prediction information from each step of diffusion models to enhance the sampling process, thereby improving the details and overall structure of generated images.
Hilbert-Guided Sparse Local Attention
Yunge Li (Oakland University), Lanyu Xu (Oakland University)
Computational EfficiencyTransformerImage
🎯 What it does: Proposed a local window/neighborhood attention based on the Hilbert curve, significantly improving the sparsity and computational efficiency of 2D image self-attention.
Hilbert: Recursively Building Formal Proofs with Informal Reasoning
Sumanth Varambally (University Of California San Diego), Ke Ye (University Of California San Diego)
AI Code AssistantTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the HILBERT framework, integrating the informal reasoning of general-purpose LLMs with the specialized Lean prover LLM, achieving formal proofs through recursive subgoal decomposition.
HiMAE: Hierarchical Masked Autoencoders Discover Resolution-Specific Structure in Wearable Time Series
Simon A. Lee (Samsung Research America), Sharanya Arcot Desai (Samsung Research America)
ClassificationGenerationConvolutional Neural NetworkTransformerAuto EncoderContrastive LearningTime SeriesBiomedical Data
🎯 What it does: Proposed and implemented a hierarchical masked autoencoder named HiMAE for pretraining on physiological time series from wearable devices, achieving downstream classification and generation tasks through multi-resolution embeddings.
Hinge Regression Tree: A Newton Method for Oblique Regression Tree Splitting
Hongyi Li (Harbin Institute of Technology), Jun Xu (Harbin Institute of Technology)
OptimizationExplainability and InterpretabilityComputational EfficiencyTabular
🎯 What it does: A new Hinge Regression Tree (HRT) is proposed, achieving more efficient and interpretable piecewise decision trees by reformulating the splitting problem at each internal node as a nonlinear least squares optimization of two linear models.
HiPO: Self-Hint Policy Optimization for RLVR
Deng Qiyuan (Harbin Institute of Technology), Zhongwen Xu (Harbin Institute of Technology)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringContrastive LearningText
🎯 What it does: This paper proposes a self-prompting strategy optimization framework called HiPO, which generates on-policy prompts using rare successful trajectories to help large language models overcome near-mistakes and exploration stagnation in RLVR tasks.
Hippoformer: Integrating Hippocampus-inspired Spatial Memory with Transformers
Tiantian Li (Qiyuan Lab), Bo Hong (Tsinghua University)
Computational EfficiencyRepresentation LearningTransformerSequential
🎯 What it does: Proposed two structured spatial memory models based on the hippocampal-entorhinal cortex theory: mm-TEM and Hippoformer, validated on multi-step prediction tasks in 2D grids and 3D random environments.
HippoTune: A Hippocampal Associative Loop–Inspired Fine-Tuning Method for Continual Learning
chenyanxi (Beijing Normal University), Xia Wu (Beijing Normal University)
ClassificationComputational EfficiencyTransformerSupervised Fine-TuningImageRetrieval-Augmented Generation
🎯 What it does: This paper proposes HippoTune, which embeds a hippocampal circuit-based iterative retrieval loop in each Transformer layer. It utilizes hidden layer states as queries, performs continuous multi-round soft retrieval with feedback updates, and simulates the pattern separation and completion mechanisms of the hippocampal EC-DG-CA3-CA1 circuit, significantly reducing catastrophic forgetting in buffer-free continual learning.
HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation
Peilin Wu (University of Texas at Dallas), Zhiyu Chen (University of Texas at Dallas)
RetrievalTransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Investigated the efficiency of retrieval behavior in Agentic Retrieval-Augmented Generation (Agentic RAG) and proposed the HiPRAG training framework, which guides the model to improve retrieval decisions through fine-grained, hierarchical process rewards.
Histopathology-Genomics Multi-modal Structural Representation Learning for Data-Efficient Precision Oncology
Kun Wu (Beihang University), Yushan Zheng
Representation LearningGraph Neural NetworkSpiking Neural NetworkTransformerContrastive LearningMultimodalityBiomedical Data
🎯 What it does: Propose a multi-modal structural representation learning framework, MSRL, which leverages graph structure learning (GSL) to capture associations between cases and uses real genomic information to assist inference when genomic data is missing.
HistoPrism: Unlocking Functional Pathway Analysis from Pan-Cancer Histology via Gene Expression Prediction
Susu Hu (National Center for Tumor Diseases), Stefanie Speidel (National Center for Tumor Diseases)
Explainability and InterpretabilityTransformerImageBiomedical Data
🎯 What it does: Proposed the HistoPrism Transformer architecture for directly predicting cross-cancer spatial gene expression from H&E pathological images, and introduced the Gene Pathway Coherence (GPC) evaluation framework.
HiTeA: Hierarchical Temporal Alignment for Training-Free Long-Video Temporal Grounding
Xinyi Xu (Nanjing University), Fang Zhao (Nanjing University)
RetrievalTransformerVision Language ModelVideoText
🎯 What it does: Propose HiTeA, a fully unsupervised and training-free long video temporal alignment framework that can localize corresponding time segments in long, uncut videos based on natural language queries.
HiVid: LLM-Guided Video Saliency For Content-Aware VOD And Live Streaming
Jiahui Chen (Tsinghua University), Lifeng Sun (Tsinghua University)
SegmentationTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Developed the HiVid framework, which uses large language models to perform content-aware weighting on video blocks and supports VOD and live streaming scenarios;
HLD: Approximate Hierarchical Linguistic Distribution Modeling for LLM-Generated Text Detection
Rui Guo (Institute of Software, Chinese Academy of Sciences), Weiming Dong (Institute of Automation, Chinese Academy of Sciences)
Anomaly DetectionExplainability and InterpretabilityComputational EfficiencyLarge Language ModelText
🎯 What it does: Proposes a framework named HLD-Detector for detecting LLM-generated text based on hierarchical language distribution, utilizing n-gram statistics and probability ratios at the word, syntactic, and semantic levels, and employing XGBoost for final judgment.
HOG-Diff: Higher-Order Guided Diffusion for Graph Generation
Yiming Huang (Imperial College London), Tolga Birdal (Imperial College London)
GenerationDrug DiscoveryGraph Neural NetworkTransformerDiffusion modelScore-based ModelGraphStochastic Differential Equation
🎯 What it does: This paper proposes a diffusion framework named HOG-Diff, based on high-order topology guidance, which adopts a coarse-to-fine curriculum learning approach to first generate the high-order skeleton of a graph and then gradually refine it into complete adjacency relations;
Holdout-Loss-Based Data Selection for LLM Finetuning via In-Context Learning
Ling Zhang (Microsoft Research Asia), Jiang Bian (Microsoft Research Asia)
Data-Centric LearningMeta LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose an ICA method based on holdout-loss for data selection and dynamic gradient reweighting in LLM fine-tuning
Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
Sayash Kapoor (Princeton University), Arvind Narayanan (Princeton University)
Large Language ModelAgentic AITextBenchmark
🎯 What it does: Developed the Holistic Agent Leaderboard (HAL) — a unified, scalable evaluation framework capable of conducting large-scale, parallel, and standardized assessments of AI agents on hundreds of VMs;
HoloPart: Generative 3D Part Amodal Segmentation
Yunhan Yang, Xihui Liu
SegmentationGenerationDiffusion modelAuto EncoderMeshBenchmark
🎯 What it does: Proposed the 3D part shape-agnostic segmentation task and provided a two-stage complete part generation process.
Homeostatic Adaptation of Optimal Population Codes under Metabolic Stress
Yi-Chun Hung (Northwestern University), Emma Alexander (Northwestern University)
Spiking Neural NetworkBiomedical Data
🎯 What it does: Developed a population coding theory integrating ATP energy and noise to explain the adaptive regulation of neurons under metabolic stress.
Horizon Imagination: Efficient On-Policy Rollout in Diffusion World Models
Lior Cohen (Technion Israel Institute Of Technology), Shie Mannor
GenerationTransformerDiffusion modelRectified FlowWorld ModelImageSequential
🎯 What it does: Proposes Horizon Imagination (HI), achieving efficient parallel on-policy imagination processes in diffusion world models, significantly reducing computational costs during inference.
Horseshoe Splatting: Handling Structural Sparsity for Uncertainty-Aware Gaussian-Splatting Radiance Field Rendering
Feng Wu (University of Hong Kong), Lequan Yu (University of Hong Kong)
GenerationComputational EfficiencyNeural Radiance FieldGaussian SplattingImage
🎯 What it does: Proposes Horseshoe Splatting, a Bayesian framework that integrates a global-local Horseshoe prior into 3D Gaussian Splatting, enabling structured sparsification of the covariance scale for each splat and achieving pixel-level uncertainty estimation.
Hot PATE: Private Aggregation of Distributions for Diverse Tasks
Edith Cohen (Google Research), Uri Stemmer (Tel Aviv University)
GenerationFederated LearningSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: Propose the Hot PATE framework, which constructs an ensemble distribution through coordinated sampling in diverse generation tasks, achieving high-quality, diverse outputs under privacy protection.
HOTA: Hamiltonian framework for Optimal Transport Advection
Nazar Buzun (Innopolis University), Dmitry V. Dylov (Computational Imaging Lab)
OptimizationReinforcement LearningImagePoint CloudBenchmarkStochastic Differential Equation
🎯 What it does: Proposed the HOTA method, which directly solves the Generalized Schrödinger Bridge using the Hamilton-Jacobi-Bellman equation and Kantarovich potential, achieving scalable trajectory optimization without explicit density modeling.
House Of Dextra : Cross-Embodied Co-Design for Dexterous Hands
Kehlani Fay (University of California San Diego), Xiaolong Wang (University of California San Diego)
OptimizationRobotic IntelligenceGraph Neural NetworkReinforcement Learning
🎯 What it does: Proposes the cross-modal co-design framework House of Dextra for rapidly generating, training, manufacturing, and deploying robotic hands that can be completed within 24 hours in simulation.
Householder-Diagonalized Linear Attention (HDLA): Utilizing Enhanced Decay Mechanism for Efficient Sequence Modeling
Jiefu Zhang (Institute of Automation, Chinese Academy of Sciences), Guoqi Li (Institute of Automation, Chinese Academy of Sciences)
ClassificationRetrievalComputational EfficiencyTransformerImageTextSequential
🎯 What it does: Proposes Householder-Diagonalized Linear Attention (HDLA), achieving a Diagonal-Plus-Rank-2 decay matrix through Householder matrix decomposition, and provides a general block-level parallel algorithm capable of supporting arbitrary rank decay and KV outer products; it is evaluated on tasks including large-scale pretraining, retrieval, zero-shot inference, and image classification.
How Catastrophic is Your LLM? Certifying Risks in Conversation
Chengxiao Wang (University of Illinois Urbana Champaign), Gagandeep Singh (University of Illinois Urbana Champaign)
Safty and PrivacyLarge Language ModelPrompt EngineeringTextGraph
🎯 What it does: Proposed a statistical authentication-based framework, C³ LLM, for catastrophic risk assessment in multi-turn dialogues with LLMs, modeling dialogues as a Markov process on a query graph, defining multiple distributions, and providing lower bounds for confidence intervals.
How Dark Patterns Manipulate Web Agents
Phil Cuvin (Stanford University), Diyi Yang (Stanford University)
Adversarial AttackLarge Language ModelAgentic AIPrompt EngineeringTextBenchmark
🎯 What it does: This study constructs the DECEPTICON environment, systematically evaluating the impact of 700 web tasks containing dark patterns on LLM-driven web agents.
How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images
Guimeng Liu (Singapore University of Technology and Design), Ngai-Man Cheung (Singapore University of Technology and Design)
RecognitionTransformerLarge Language ModelVision Language ModelMultimodalityBiomedical DataBenchmark
🎯 What it does: This paper systematically analyzes the visual localization capabilities of medical multimodal large language models in medical visual question answering, and on this basis designs a specialized evaluation dataset VGMED and an attention refinement method VGRefine that requires no training.
How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability
Shawn Im (University of Wisconsin Madison), Sharon Li (University of Wisconsin Madison)
Explainability and InterpretabilityTransformerText
🎯 What it does: This paper investigates how Transformers form semantic associations during natural language data training and provides a closed-form gradient-dominated approximation for the weight matrix.
How does the optimizer implicitly bias the model merging loss landscape?
Chenxiang Zhang (University of Luxembourg), Sjouke Mauw (University of Luxembourg)
OptimizationImageText
🎯 What it does: Investigate how the implicit bias of optimizers determines the feasibility and effectiveness of model merging, and propose the effective noise scale as a unified metric.
How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Use
Minhua Lin (Pennsylvania State University), Suhang Wang (Pennsylvania State University)
TransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringText
🎯 What it does: This study systematically evaluates the performance of large language models (LLMs) in imperfect information games such as poker, deeply analyzes their reasoning flaws, and proposes the ToolPoker framework, which integrates external poker solvers into the LLM's reasoning process to enhance strategy and reasoning quality.
How Far Can Unsupervised RLVR Scale LLM Training?
Bingxiang He (Tsinghua University), Ning Ding (Tsinghua University)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper systematically evaluates and theorizes the scalability of Unsupervised Verifiable Reinforcement (URLVR) in large language models (LLMs), focusing on comparing intrinsic and extrinsic reward methods. It proposes the 'Model Collapse Step' as a diagnostic metric to assess model priors and verifies the safety of intrinsic rewards in small-scale data and test-time training.
How hard is learning to cut? Trade-offs and sample complexity
Sammy Khalife (Cornell University), Andrea Lodi (Technion)
OptimizationGraph Neural NetworkBenchmark
🎯 What it does: Propose a lower bound on sample complexity under the learning-to-cut framework, and experimentally verify that gap closed can serve as an effective proxy for tree size.
How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
Kairong Luo (Tsinghua University), Wenguang Chen (Tsinghua University)
OptimizationRepresentation LearningHyperparameter SearchData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Studies the contradiction between learning rate decay and data curriculum (sorted by quality) in pre-training large language models, and proposes using moderate learning rate decay combined with weighted averaging (CMA/CDMA) under curriculum to enhance model performance.
How Many Code and Test Cases Are Enough? Evaluating Test Cases Generation from a Binary-Matrix Perspective
Xianzhen Luo (Harbin Institute of Technology), Wanxiang Che (Harbin Institute of Technology)
AI Code AssistantLarge Language ModelTextBenchmark
🎯 What it does: Proposed an evaluation framework based on the rank of binary matrices, designed an approximate algorithm called WrongSelect, and constructed the TC-Bench benchmark based on competition submissions.
How Muon’s Spectral Design Benefits Generalization: A Study on Imbalanced Data
Bhavya Vasudeva (University of Southern California), Christos Thrampoulidis (University of British Columbia)
OptimizationData-Centric LearningImageText
🎯 What it does: Conduct theoretical and experimental studies on spectral-aware optimizers (Muon, Shampoo), demonstrating that the canonical form SpecGD can learn all principal components faster on imbalanced data, thereby improving early generalization performance.
How NOT to benchmark your SITE metric: Beyond Static Leaderboards and Towards Realistic Evaluation.
Prabhant Singh (AMOR/e Lab TU Eindhoven), Joaquin Vanschoren (AMOR/e Lab TU Eindhoven)
ClassificationImageBenchmark
🎯 What it does: This paper systematically evaluates existing Source Independent Transferability Estimation (SITE) assessment benchmarks, revealing issues such as unrealistic model spaces, leaderboards dominated by static models, and lack of consideration for the authenticity of score differences. It proposes four best practices and an improved diversified benchmark, verifying that existing SITE metrics perform unstably after these improvements.
How reinforcement learning after next-token prediction facilitates learning
Nikolaos Tsilivis (New York University), Julia Kempe (New York University)
TransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Studied the training process in large language models where next-word prediction (pre-training) is followed by reinforcement learning (post-training), and analyzed theoretically and experimentally how reinforcement learning significantly enhances model generalization under data distributions with a small number of long-chain-of-thought examples.
How Reliable is Language Model Micro-Benchmarking?
Gregory Yauney, Swabha Swayamdipta
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose a novel micro-benchmark evaluation metric called MDAD (Minimum Detectable Ability Difference) to measure the reliability of micro-benchmarks in ranking predictive models; simultaneously conduct systematic experimental comparisons of multiple existing micro-benchmark methods;
How Stable is the Next Token? A Geometric View of LLM Prediction Stability
Deyuan Liu (Harbin Institute of Technology), Dianbo Sui (Harbin Institute of Technology)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose Token Constraint Bound (δ TCB) to quantify the local prediction stability of large language models (LLMs) against perturbations in internal hidden states under a given context.
How Text Quality Interventions Reshape Neural Scaling Laws for LLMs: Empirical Study
Newsha Ardalani (FAIR at Meta), Carole-Jean Wu (FAIR at Meta)
Computational EfficiencyData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Conduct a large-scale empirical study on text quality interventions (deduplication, heuristic filtering, LLM rewriting), training over 2000 models and proposing the QualityPajama benchmark.
How to Cure Newton for Unlearning Neural Networks? An Empirical Study from the Hessian Perspective
Nhung Bui (National University of Singapore), Bryan Kian Hsiang Low (National University of Singapore)
OptimizationComputational EfficiencyImageText
🎯 What it does: Proposed two novel second-order dropout algorithms, CuReNU and CuReNUS, utilizing cubic regularization to automatically adjust the Hessian matrix damping, addressing the issue of Newton dropout failure caused by Hessian matrix degeneration during training of neural networks and large language models.
How to Lose Inherent Counterfactuality in Reinforcement Learning
Ezgi Korkmaz
Adversarial AttackReinforcement LearningVideo
🎯 What it does: Through theoretical derivation and experimental validation, comparing the impact of standard reinforcement learning and ε-local invariance robust training, it is found that the latter causes value function distortion, adversarial degradation, and loss of intrinsic counterfactual reasoning ability.
How to Square Tensor Networks and Circuits Without Squaring Them
Lorenzo Loconte (University of Edinburgh, UK), Antonio Vergari (University of Edinburgh, UK)
Computational EfficiencyRepresentation LearningImage
🎯 What it does: This paper proposes a new framework that leverages orthogonality and unitary orthogonalization conditions, enabling efficient marginalization and normalization of squared tensor networks and squared probabilistic circuits without explicitly squaring them.
How to train data-efficient LLMs
Noveen Sachdeva (Google DeepMind), Derek Zhiyuan Cheng (Google DeepMind)
Data-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes two efficient pre-training data sampling methods—ASK-LLM (utilizing instruction-tuned LLMs to directly evaluate data quality through prompts) and DENSITY (coverage sampling based on density estimation), and verifies their effectiveness through large-scale experiments.
How Transformers Learn Causal Structures In-Context: Explainable Mechanism Meets Theoretical Guarantee
Jianzhe Wei (Georgia Institute of Technology), Zhuoran Yang (Yale University)
Explainability and InterpretabilityTransformerSequential
🎯 What it does: Examines how Transformers infer hidden causal structures in in-context learning tasks without parameter updates, and demonstrates their ability to achieve Bayesian Model Averaging (BMA) inference.
How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Tasks
Rahul Ramachandran (Swiss Federal Institute Of Technology), Amir Zamir (Swiss Federal Institute Of Technology)
ClassificationObject DetectionSegmentationDepth EstimationTransformerPrompt EngineeringImageMultimodalityBenchmark
🎯 What it does: This study proposes a framework based on prompt chaining that leverages multimodal foundation models (MFM) to perform traditional computer vision tasks (semantic segmentation, object detection, image classification, depth and normal prediction, etc.) through text interaction.
HSG-12M: A Large-Scale Benchmark of Spatial Multigraphs from the Energy Spectra of Non-Hermitian Crystals
Xianquan Yan (National University of Singapore), Ching Hua Lee (National University of Singapore)
Data SynthesisRepresentation LearningGraph Neural NetworkGraphBenchmarkPhysics Related
🎯 What it does: Built the Poly2Graph automated pipeline, generating 12 million Hamiltonian spectral graphs and forming the HSG-12M dataset, and evaluated multiple GNNs on this dataset.
HSIC Bottleneck for Cross-Generator and Domain-Incremental Synthetic Image Detection
Chin-Chia Yang (National Taiwan University), Tyng-Luh Liu (Academia Sinica)
Object DetectionData SynthesisDomain AdaptationTransformerGaussian SplattingImagePoint Cloud
🎯 What it does: Study cross-generator and domain incremental synthetic image detection, proposing HSIC bottleneck and HSIC-Guided Replay.
HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Models
Zhaolu Kang (Peking University), Xuelong Li (China Telecom)
TransformerPrompt EngineeringMultimodalityBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed HSSBench — a multimodal large language model evaluation benchmark focused on the humanities and social sciences field;
Hubble: a Model Suite to Advance the Study of LLM Memorization
Johnny Wei, Robin Jia (University of Southern California)
Safty and PrivacyTransformerLarge Language ModelText
🎯 What it does: Propose the HUBBLE suite, which includes standard and perturbed versions of large language models, for systematically studying memory risks in LLMs.
Human Behavior Atlas: Benchmarking Unified Psychological And Social Behavior Understanding
Keane Ong (Massachusetts Institute Of Technology), Paul Pu Liang (Massachusetts Institute Of Technology)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringMultimodalityBenchmark
🎯 What it does: Built a unified psychological and social behavior benchmark, HUMAN BEHAVIOR ATLAS, containing over 100,000 multimodal samples across multiple dimensions, including emotion, cognition, disease, and social processes, and trained three 7B-scale multimodal LLMs (SFT, BAM, RL) on this benchmark.
Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction
Xiang Li (Beijing University of Posts and Telecommunications), Benyou Wang (Shenzhen Research Institute of Big Data)
ClassificationExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelTextAudio
🎯 What it does: Conducted the first Turing test on existing 9 speech-to-speech (S2S) systems and constructed a dataset of 1,486 dialogues containing human-machine, real human, and pseudo-human (TTS synthesized) dialogues; subsequently designed an 18-dimensional fine-grained human similarity classification system to evaluate dialogues; finally developed an interpretable AI judgment model based on this classification system to automatically distinguish human-machine dialogues and provide fine-grained diagnostics.
Human Uncertainty-Aware Data Selection and Automatic Labeling in Visual Question Answering
Jian Lan (University of Munich), Thomas Seidl (University of Munich)
Data-Centric LearningSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: Propose HaDola, a data selection and automatic annotation framework based on human uncertainty (HU), which can efficiently fine-tune visual question answering (VQA) models with only 5% HU-annotated data.
Human-LLM Collaborative Feature Engineering for Tabular Data
Zhuoyan Li (Purdue University), Yunyao Li (Adobe)
OptimizationData-Centric LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelTabularBenchmark
🎯 What it does: This study proposes a human-LLM collaborative feature engineering framework. It first allows the LLM to generate diverse feature transformation candidates, then models the utility and uncertainty of each candidate using a Bayesian neural network, selects using UCB, and selectively acquires human expert preference feedback when necessary to improve decision-making.
Human-MME: A Holistic Evaluation Benchmark for Human-Centric Multimodal Large Language Models
Yuansen Liu (National University Of Singapore), Shuicheng YAN
ClassificationRecognitionObject DetectionSegmentationLarge Language ModelVision Language ModelImageMultimodalityBenchmark
🎯 What it does: Constructed the Human-MME benchmark to comprehensively evaluate the fine-grained perception and high-level causal reasoning capabilities of multimodal large language models (MLLMs) on human-centric images.
Human-Object Interaction via Automatically Designed VLM-Guided Motion Policy
Zekai Deng (ShanghaiTech University), Jingya Wang (ShanghaiTech University)
Data SynthesisRobotic IntelligenceTransformerReinforcement LearningVision Language ModelMeshSequentialBenchmark
🎯 What it does: Propose a unified physics-based HOI framework that leverages vision-language models to automatically generate goal states and reward functions, enabling natural action generation for long-term, multi-object interactions.
Human3R: Everyone Everywhere All at Once
Yue Chen (Zhejiang University), Gerard Pons-Moll (University of Tübingen)
Object TrackingSegmentationPose EstimationDepth EstimationComputational EfficiencyTransformerSupervised Fine-TuningPrompt EngineeringVideoMeshBenchmark
🎯 What it does: This study proposes a unified, real-time online 4D human and scene reconstruction framework called Human3R, which can simultaneously recover the global SMPL-X human mesh, dense 3D scenes, and camera trajectories within a single forward pass, supporting online inference for multi-human scenarios.
Humanline: Online Alignment as Perceptual Loss
Sijia Liu (Princeton University), Kawin Ethayarajh (University of Chicago)
Computational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Study online and offline alignment methods, propose an explanation of the advantages of online methods from a human perception perspective based on prospect theory, and design a universal pattern that can convert any alignment objective (DPO, KTO, GRPO) into a 'humanline' version.
HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes
Keliang Li (Institute of Computing Technology, Chinese Academy of Sciences), Xilin Chen (Institute of Computing Technology, Chinese Academy of Sciences)
TransformerLarge Language ModelImageVideoMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Propose the HumanPCR evaluation kit to conduct fine-grained assessment of multimodal large language models (MLLMs) in diverse human-centric visual scenarios, focusing on their perception, understanding, and reasoning capabilities;
HUME: Measuring the Human-Model Performance Gap in Text Embedding Tasks
Adnan El Assadi (Carleton University), Kenneth Enevoldsen (Aarhus University)
ClassificationRetrievalRepresentation LearningLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes the HUME framework, conducting human evaluations on 16 text embedding tasks in MTEB and providing a human performance benchmark.
HUMOF: Human Motion Forecasting in Interactive Social Scenes
Caiyi Sun (ShanghaiTech University), Yuexin Ma (ShanghaiTech University)
Pose EstimationGraph Neural NetworkTransformerPoint Cloud
🎯 What it does: Propose a human motion prediction method called HUMOF, which integrates hierarchical feature representations of human-human and human-environment interactions, along with a coarse-to-fine scale interaction reasoning module for complex dynamic interaction scenarios.
Huxley-G\"odel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine
Wenyi Wang (King Abdullah University of Science and Technology), Jürgen Schmidhuber (King Abdullah University of Science and Technology)
OptimizationMeta LearningAI Code AssistantLarge Language ModelText
🎯 What it does: Proposed a self-improving evaluation metric called CMP based on line families, and constructed the Huxley-Gödel Machine (HGM) guided by CMP, achieving performance comparable to humans in the automation of software engineering coding agents.
HWC-Loco: A Hierarchical Whole-Body Control Approach to Robust Humanoid Locomotion
Sixu Lin (Chinese University of Hong Kong), Guiliang Liu (Chinese University of Hong Kong)
OptimizationRobotic IntelligenceReinforcement LearningAuto EncoderGenerative Adversarial NetworkTime Series
🎯 What it does: Propose a hierarchical whole-body control framework named HWC-LOCO, enabling humanoid robots to achieve safe and robust locomotion on diverse terrains and under disturbance environments.
Hybrid Deep Searcher: Scalable Parallel and Sequential Search Reasoning
Dayoon Ko (Seoul National University), Kyungjae Lee (LG AI Research)
RetrievalTransformerSupervised Fine-TuningReinforcement LearningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Designed and trained HybridDeepSearcher, a model capable of performing parallel queries and sequential reasoning simultaneously within a retrieval-augmented generation framework to achieve scalable search inference.
Hybrid Reinforcement: when reward is sparse, better to be dense
Leitian Tao (FAIR at Meta), Ping Yu (FAIR at Meta)
Reinforcement Learning from Human FeedbackTransformerReinforcement LearningPrompt EngineeringContrastive LearningText
🎯 What it does: Proposes the HERO framework, which combines verifiable 0/1 rewards with continuous reward models for tasks requiring verifiable results, such as mathematical reasoning.
Hybrid Training for Vision-Language-Action Models
Pietro Mazzaglia (Qualcomm AI Research), Daniel Dijkman (Qualcomm AI Research)
Robotic IntelligenceTransformerVision Language ModelVision-Language-Action ModelMultimodalityChain-of-Thought
🎯 What it does: Propose the Hybrid Training (HyT) framework, enabling Vision-Language-Action (VLA) models to learn Chain-of-Thought (CoT) reasoning during training, while directly outputting actions during inference, balancing performance improvement with high-speed inference.
HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model
Jiaming Liu (Peking University), Shanghang Zhang (Peking University)
Robotic IntelligenceTransformerLarge Language ModelVision Language ModelVision-Language-Action ModelDiffusion modelMultimodality
🎯 What it does: Propose the HybridVLA model, unifying autoregressive and diffusion generation methods within a single large language model (LLM), enabling robot vision-language-action prediction;
Hyden: A Hybrid Dual-Path Encoder for Monocular Geometry of High-resolution Images
Zaiwei Zhang (Meta Reality Labs), JQ Huang (Meta Reality Labs)
Depth EstimationKnowledge DistillationConvolutional Neural NetworkTransformerImage
🎯 What it does: Propose the Hyden hybrid dual-path encoder, using Vision Transformer to extract global context and convolutional networks to preserve details, achieving monocular geometry estimation at high resolution.
Hyper-SET: Designing Transformers via Hyperspherical Energy Minimization
Yunzhe Hu (University of Hong Kong), Dong Xu (University of Hong Kong)
ClassificationRestorationTransformerImage
🎯 What it does: This paper proposes a Transformer architecture HYPER-SET that shares parameters and can be recursively expanded by minimizing the energy function on a hypersphere to achieve distribution uniformity and semantic alignment.
HYPER: A Foundation Model for Inductive Link Prediction with Knowledge Hypergraphs
Xingyue Huang (University of Oxford), Ismail Ilkan Ceylan (TU Wien)
Representation LearningGraph Neural NetworkGraphBenchmark
🎯 What it does: Proposes HYPER—a foundational model for inductive link prediction on knowledge hypergraphs with arbitrary arity, supporting zero-shot reasoning for new entities and relations.
Hyperbolic Aware Minimization: Implicit Bias for Sparsity
Tom Jacobs (CISPA Helmholtz Center for Information Security), Rebekka Burkholz (CISPA Helmholtz Center for Information Security)
ClassificationOptimizationImageGraph
🎯 What it does: Propose a lightweight optimization step called 'Hyperplane-Aware Minimization' (HAM), combining traditional gradient descent with exponential mirror steps to enhance generalization performance in both sparse and dense training.
Hyperparameter Trajectory Inference with Conditional Lagrangian Optimal Transport
Harry Amad (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
OptimizationHyperparameter SearchDiffusion modelTime SeriesBiomedical DataBenchmark
🎯 What it does: Proposes the Hyperparameter Trajectory Inference (HTI) task, learning the conditional output distribution changes of neural networks under different hyperparameters, and constructing a proxy model capable of dynamically adjusting hyperparameters during inference.
Hyperspherical Latents Improve Continuous-Token Autoregressive Generation
Guolin Ke (DP Technology), HUI XUE
GenerationTransformerDiffusion modelAuto EncoderImage
🎯 What it does: Designed and implemented SphereAR, a continuous-token image generation framework based on hyperspherical VAE and autoregressive Transformer, addressing the variance collapse problem in traditional AR models during decoding.
Hystar: Hypernetwork-driven Style-adaptive Retrieval via Dynamic SVD Modulation
Yujia Cai (Xidian University), Jiexi Yan (Xidian University)
RetrievalTransformerVision Language ModelContrastive LearningImage
🎯 What it does: Propose a dynamic parameter-efficient fine-tuning framework called Hystar based on hypernetworks, combining singular value decomposition (SVD) modulation with static offsets, and introducing an OT-weighted contrastive loss called StyleNCE to achieve query image retrieval under diverse style queries.
I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
Yuhang Liu (Responsible AI Research Centre), Javen Qinfeng Shi (Responsible AI Research Centre)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelAuto EncoderTextSequential
🎯 What it does: This paper proposes a discrete latent variable model, proving that in large language models trained with next-word prediction, internal representations are approximately linear transformations of the logarithm of the latent variable posterior; it further unifies the explanation of the linear representation hypothesis; based on this theory, a framework is designed to evaluate sparse autoencoders (SAE) and structured SAE, utilizing controlled experiments for linear probing of concepts.
I-DRUID: Layout to image generation via instance-disentangled representation and unpaired data
Fengxiang Yang (vivo Mobile Communication Co., Ltd.), Bo Li (vivo Mobile Communication Co., Ltd.)
GenerationConvolutional Neural NetworkReinforcement LearningDiffusion modelImageTextStochastic Differential Equation
🎯 What it does: Proposed a layout-to-image generation framework called I-DRUID that integrates instance disentangled representation with unpaired data.
I2Mole: Interaction-aware Invariant Molecular Learning For Generalizable Property Prediction
Wenjie Du (University of Science and Technology of China), Yang Wang (University of Science and Technology of China)
Drug DiscoveryGraph Neural NetworkGraphBiomedical DataBenchmark
🎯 What it does: Proposes an interaction-aware invariant molecular relationship learning framework named I2Mole for drug-drug interaction (DDI) prediction, which uses a merged graph to finely model atomic interactions between molecular pairs and extracts core substructures through an improved graph information bottleneck (GIB).
IA2: Alignment with ICL Activations improves Supervised Fine-Tuning
Aayush Mishra (Johns Hopkins University), Anqi Liu (Johns Hopkins University)
Knowledge DistillationRepresentation LearningTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: An improvement over traditional supervised fine-tuning (SFT) is proposed, where the model first aligns its internal activation space with the In-Context Learning (ICL) performance during inference through self-distillation (IA2), followed by standard SFT to enhance model accuracy and calibration.
IC-Custom: Diverse Image Customization via In-Context Learning
Yaowei Li (Peking University), Ying Shan (Tencent)
GenerationTransformerSupervised Fine-TuningDiffusion modelFlow-based ModelImageTextMultimodality
🎯 What it does: Propose a unified image customization framework called IC-Custom, which can simultaneously handle both position-aware and position-agnostic image customization tasks.