π― What it does: Proposes RetroAttention, a KV cache compression technique for long-text generation that compresses KV caches by backward correction of attention outputs.
ReVeal: Self-Evolving Code Agents via Reliable Self-Verification
Yiyang Jin (Tongji University), Jing Bai (Microsoft Research Asia)
CodeAI Code AssistantTransformerLarge Language ModelReinforcement LearningAgentic AIText
π― What it does: Proposed a multi-round reinforcement learning framework called ReVeal, enabling large language models to self-verify and continuously improve while generating code, ultimately achieving self-evolution of code.
Revela: Dense Retriever Learning via Language Modeling
Fengyu Cai (Technical University of Darmstadt), Heinz Koeppl (Technical University of Darmstadt)
CodeRetrievalTransformerLarge Language ModelContrastive LearningText
π― What it does: Trained a dense retriever by jointly optimizing retrieval and generation through self-supervised language modeling (next-word prediction).
Reverse Distillation: Consistently Scaling Protein Language Model Representations
Darius Catrina (Duke University), Rohit Singh (Flatiron Institute)
CodeKnowledge DistillationRepresentation LearningTransformerLarge Language ModelAuto EncoderBiomedical Data
π― What it does: Decompose the representations of large protein language models into orthogonal subspaces with small models as a base through reverse distillation, constructing Matryoshka-style embeddings to achieve consistent scaling improvements.
Revisit Visual Prompt Tuning: The Expressiveness of Prompt Experts
Minh Le (Trivita AI), Nhat Ho (University of Texas at Austin)
CodeClassificationSegmentationTransformerPrompt EngineeringMixture of ExpertsContrastive LearningImage
π― What it does: Proposed Visual Adaptive Prompt Tuning (VAPT), significantly enhancing visual model adaptation performance by introducing dynamic prompt experts based on VPT.
Revisiting Long-context Modeling from Context Denoising Perspective
Zecheng Tang (Soochow University), Min Zhang (Soochow University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Propose Context Denoising Training (CDT), which detects and suppresses irrelevant noise in the input, enabling long-context models to better focus on key information.
Revisiting Multimodal Positional Encoding in VisionβLanguage Models
Jie Huang (Qwen Team, Alibaba Group), Shuai Bai (Qwen Team, Alibaba Group)
CodeTransformerSupervised Fine-TuningVision Language ModelImageVideoTextMultimodality
π― What it does: This work conducts a systematic analysis of multimodal RoPE position encoding, proposes two lightweight plug-and-play schemes (Multi-Head RoPE and MRoPE-Interleave), and introduces a spatial reset mechanism to enhance visual information processing;
Revisiting Nonstationary Kernel Design for Multi-Output Gaussian Processes
Qiaochu Xu (University of Hong Kong), Pablo M. Olmos (University Carlos III de Madrid)
CodeTabularTime Series
π― What it does: Proposed and implemented a multi-output low-rank non-stationary kernel (MO-LRN) to enhance non-stationary modeling in multi-output Gaussian processes (MOGP).
Xinyi Wan (Sea AI Lab), Jialin Li (National University Of Singapore)
CodeComputational EfficiencyLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: Revisit the parameter server model, integrate it into Fully Sharded Data Parallel (FSDP), and propose On-Demand Communication (ODC) to reduce synchronization bottlenecks and improve the throughput of LLM post-training.
Revisiting Sharpness-Aware Minimization: A More Faithful and Effective Implementation
Jianlong Chen (Shanghai University of Finance and Economics), Zhiming Zhou (Shanghai University of Finance and Economics)
CodeOptimizationTransformerImageText
π― What it does: Reinterpret and improve Sharpness-Aware Minimization (SAM), proposing a more accurate and adaptive direction estimation method called XSAM.
Revisiting the Scaling Properties of Downstream Metrics in Large Language Model Training
Jakub Krajewski (University of Warsaw), Jason Ramapuram (Apple)
CodeComputational EfficiencyTransformerLarge Language ModelText
π― What it does: This paper proposes a power-law scaling model that directly predicts the accuracy of downstream tasks for large language models from the training budget (FLOPs);
Revisiting Tree-Sliced Wasserstein Distance Through the Lens of the FermatβWeber Problem
Viet-Hoang Tran (National University of Singapore), Tan Minh Nguyen (National University of Singapore)
CodeOptimizationRepresentation LearningImageText
π― What it does: This paper proposes the Fermat-Weber Tree-Sliced Wasserstein (FW-TSW) method, improving the sampling strategy of Tree-Sliced Wasserstein by utilizing the geometric median and Weiszfeld algorithm to determine tree root points and directions, thereby better capturing the spatial structure of distributions.
π― What it does: This paper investigates the application of weight regularization in low-rank parameterized continual learning and proposes a novel EWC-LoRA method;
Revisting Node Affinity Prediction In Temporal Graphs
Or Feldman (Ben-Gurion University of the Negev), Chaim Baskin (Ben-Gurion University of the Negev)
CodeGraph Neural NetworkGraphTime SeriesBenchmark
π― What it does: Proposed a node affinity prediction framework for temporal graphs called NAVIS, and experimentally validated its superiority over existing TGNNs and simple heuristic methods.
Revisual-R1: Advancing Multimodal Reasoning From Optimized Cold Start to Staged Reinforcement Learning
Shuang Chen, Yu Cheng (Chinese University of Hong Kong)
CodeOptimizationTransformerLarge Language ModelReinforcement LearningVision Language ModelTextMultimodality
π― What it does: Proposed ReVisual-R1, which integrates text-optimized cold start, multi-modal reinforcement learning, and text reinforcement learning in a three-stage training process, enhancing the inference performance of 3B/7B multi-modal large language models.
Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models
Yinjie Wang (Princeton University), Mengdi Wang (Princeton University)
CodeAI Code AssistantTransformerLarge Language ModelReinforcement LearningDiffusion modelTextBenchmark
π― What it does: This paper proposes TraceRLβa trajectory-aware reinforcement learning framework that combines diffusion value models to enhance the performance of full-attention and block-attention diffusion language models on reasoning, mathematics, and code tasks, and generates the TraDo series of state-of-the-art models.
Xinle Wu (National University of Singapore), Yao Lu (National University of Singapore)
CodeReinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement LearningTextBenchmark
π― What it does: Propose BayesianRouter, a hybrid routing framework combining offline RM strength learning with online Bayesian Thompson sampling, used for dynamically selecting reward models in RLHF.
Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models
David Bani-Harouni (Technical University of Munich), Matthias Keicher (Technical University of Munich)
CodeExplainability and InterpretabilityReinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningText
π― What it does: Propose a reward mechanism based on reinforcement learning, enabling large language models to provide calibrated confidence scores when answering questions;
Rex-Thinker: Grounded Object Referring via Chain-of-Thought Reasoning
Qing Jiang (South China University of Technology), Lei Zhang (International Digital Economy Academy)
CodeObject DetectionExplainability and InterpretabilityLarge Language ModelSupervised Fine-TuningReinforcement LearningVision Language ModelMultimodalityChain-of-Thought
π― What it does: Propose a multimodal large model framework named REX-THINKER based on Chain-of-Thought (CoT) for object reference tasks, capable of providing interpretable and verifiable predictions and refusing to output when no matching object is present.
π― What it does: Constructed and publicly released the first large-scale, wideband, and geometrically diverse RF material identification dataset, RF-MatID, and established multi-band protocols and benchmark evaluations across angles/distances on it.
RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models
Yunseok Han (Seoul National University), Jaeyoung Do (Seoul National University)
CodeExplainability and InterpretabilityLarge Language ModelTextBenchmark
π― What it does: This paper proposes RFEval, a framework for evaluating the reasoning trustworthiness of large-scale reasoning models, and constructs a benchmark dataset with 7,186 instances under this framework;
π― What it does: This paper proposes a new algorithm called RFedAGS for federated learning on Riemannian manifolds, which can be trained under conditions of partial participation and data heterogeneity.
RiskPO: Risk-based Policy Optimization with Verifiable Reward for LLM Post-Training
Tao Ren (Peking University), Yijie Peng (Peking University)
CodeOptimizationLarge Language ModelReinforcement LearningTextMultimodality
π― What it does: Propose the RiskPO framework for post-training of LLMs, employing a risk-sensitive objective MVaR (Conditional Value at Risk) for distributed optimization of verifiable rewards.
RIVER: A Real-Time Interaction Benchmark for Video LLMs
Yansong Shi (University of Science and Technology of China), Limin Wang (Nanjing University)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoBenchmark
π― What it does: Constructed a real-time video interaction benchmark named RIVER Bench to evaluate the capabilities of video large language models (LLMs) in retro memory, real-time perception, and proactive response, and proposed a framework to enhance online inference along with specialized online training data.
RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?
Yiyou Sun, Dawn Song (University Of California Berkeley)
CodeTransformerLarge Language ModelReinforcement LearningText
π― What it does: Designed and constructed a controlled programming problem set named DELTA, and trained LLMs using reinforcement learning (RL) to investigate the feasibility of learning new reasoning strategies and their transferability.
CodeClassificationSegmentationRepresentation LearningData-Centric LearningReinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement LearningMultimodality
π― What it does: This paper systematically compares the effects of supervised fine-tuning (SFT) and reinforcement learning (RL, using DPO) on multi-modal language models (MLLMs) and their visual encoders, and proposes a training scheme called Preference-Instructed Vision OpTimization (PIVOT) based on the advantages of RL. This scheme optimizes the visual encoder using RL, significantly improving visual representations and downstream task performance.
RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning
Qianyue Hao (Tsinghua University), Yong Li (Tsinghua University)
CodeLarge Language ModelReinforcement LearningTextChain-of-Thought
π― What it does: Propose the RL-of-Thoughts (RLoT) framework, which uses reinforcement learning to train a lightweight navigator that dynamically generates task-specific logical structures during inference to enhance the reasoning capabilities of large language models (LLMs).
CodeRepresentation LearningTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
π― What it does: Propose an RLP (Reinforcement Learning Pretraining) method for improving large language models by rewarding Chain-of-Thought information gain during the pretraining phase.
π― What it does: Proposed and implemented the RLVER framework, which utilizes verifiable sentiment rewards to train large language models, enabling them to possess higher-order empathy and emotional support capabilities.
Xiusi Chen (University of Illinois at Urbana-Champaign), Heng Ji (University of Illinois at Urbana-Champaign)
CodeExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextChain-of-Thought
π― What it does: This paper proposes a framework that treats reward modeling as an inference taskβReasoning Reward Models (RM-R1). By employing a two-phase training process (reasoning distillation + reinforcement learning with verifiable rewards), the model first generates long-chain reasoning and evaluation criteria before assigning reward scores, thereby enhancing the interpretability and performance of reward models.
π― What it does: Proposes the RMAAT (Recurrent Memory Augmented Astromorphic Transformer) architecture, abstracting the long-term and short-term plasticity mechanisms of astrocytes in processing long sequences into memory compression and linear attention, addressing the quadratic complexity bottleneck of Transformers on long sequences.
RoboMD: Uncovering Robot Vulnerabilities through Semantic Potential Fields
Som Sagar (Arizona State University), Ransalu Senanayake (Arizona State University)
CodeRobotic IntelligenceTransformerReinforcement LearningVision Language ModelContrastive LearningMultimodality
π― What it does: Developed a diagnostic framework called RoboMD based on deep reinforcement learning to actively search for failure modes in robot manipulation strategies.
RoboOmni: Proactive Robot Manipulation in Omni-modal Context
Siyin Wang (Fudan University), Xipeng Qiu (Fudan University)
CodeRobotic IntelligenceTransformerLarge Language ModelVision-Language-Action ModelMultimodalityBenchmark
π― What it does: This study proposes RoboOmni, an end-to-end full-modal large language model based on the Perceiver-Thinker-Talker-Executor architecture, capable of proactively inferring user intent from cross-modal context instructions (voice, environmental sounds, visual) and completing confirmation and action execution; meanwhile, it constructs the OmniAction dataset containing 140k multimodal tasks, 5k+ speakers, 2.4k event audios, and 640 background sounds, and generates OmniAction-LIBERO simulation and real environment evaluations based on this dataset.
RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Tasks
Shiying Duan (Beihang University), wenjun wu
CodeRobotic IntelligenceTransformerLarge Language ModelTextGraphBenchmarkRetrieval-Augmented Generation
π― What it does: Proposes RoboPARA, a dual-arm robot task parallel planning framework based on large language models, and designs a new dataset X-DAPT for evaluating dual-arm parallel planning.
CodeDomain AdaptationRobotic IntelligenceReinforcement Learning from Human FeedbackVision Language ModelVideoBenchmark
π― What it does: Propose the RobotArena β framework, which automatically converts real robot videos into simulation environments and evaluates general robot policies in these digital twins.
Robust Adversarial Attacks Against Unknown Disturbance via Inverse Gradient Sample
Zhaoyang Zhang (Harbin Institute of Technology), Yihan Yan (Harbin Institute of Technology)
CodeAdversarial AttackImage
π― What it does: Propose a new robust adversarial attack framework called IGSA, which uses inverse gradient sampling to identify the most destructive direction in the perturbation space and iteratively optimizes adversarial examples to maintain high attack success rates when facing unknown perturbations.
Robust Adversarial Quantification via Conflict-Aware Evidential Deep Learning
Charmaine Barker (University of York), Simos Gerasimou (Cyprus University of Technology)
CodeAnomaly DetectionAdversarial AttackImage
π― What it does: Perform post-hoc uncertainty calibration on pre-trained Evidential Deep Learning (EDL) models by generating multi-perspective evidence through label-preserving transformations, and adjust the evidence based on conflict measures to enhance detection of out-of-distribution (OOD) samples and adversarial examples.
π― What it does: Propose a semi-supervised neural amortized Bayesian inference method that trains on unlabelled data using a self-consistency loss, thereby enhancing inference robustness in out-of-distribution scenarios.
Robust Equation Structure Learning with Adaptive Refinement
Yunlun Li (The Chinese University of Hong Kong), Sinno Jialin Pan (The Chinese University of Hong Kong)
CodeOptimizationTransformerLarge Language ModelPrompt EngineeringTextBenchmarkPhysics Related
π― What it does: Proposes the RESTART framework, fully realizing the hypothesis-experiment-analysis cycle in scientific discovery, achieving adaptive structural learning and improvement in symbolic regression
CodeFederated LearningSafty and PrivacyAdversarial AttackImageText
π― What it does: Proposed and studied the robustness issue in federated inference scenarios, providing security analysis and defense schemes for average aggregation and nonlinear aggregation (DeepSet) when up to f/2 clients are attacked.
π― What it does: Investigated the suboptimal transfer phenomenon that occurs when performing robust fine-tuning from non-robust pre-trained models, and proposed Epsilon-Scheduling to alleviate it;
Robust LLM Unlearning via Post Judgment and Multi-round Thinking
Xinrui Chen (Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences), Ou Wu (Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences)
CodeSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: Propose the PoRT framework to achieve zero-shot learning in LLMs and significantly enhance robustness.
Robust Optimization for Mitigating Reward Hacking with Correlated Proxies
Zixuan Liu (Tulane University), Zizhan Zheng (Tulane University)
CodeOptimizationReinforcement Learning from Human FeedbackReinforcement LearningBenchmark
π― What it does: This paper proposes a robust max-min strategy optimization framework to prevent reward hacking when there is an r-relatedness between the reward proxy and the true reward; meanwhile, it provides an interpretable version for linear reward structures;
Robust Preference Alignment via Directional Neighborhood Consensus
Ruochen Mao (Hong Kong University of Science and Technology (Guangzhou)), Jiaheng Wei (Hong Kong University of Science and Technology (Guangzhou))
CodeReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningText
π― What it does: Proposed a Robust Preference Selection (RPS) method in the post-inference phase, leveraging directional neighborhood consistency to sample multiple candidate responses near the user preference vector and select the optimal one to enhance robustness in preference alignment.
Robust Test-time Video-Text Retrieval: Benchmarking and Adapting for Query Shifts
Bingqing Zhang (University of Queensland), Sen Wang (University of Queensland)
CodeRetrievalDomain AdaptationVision Language ModelContrastive LearningVideoTextMultimodalityBenchmark
π― What it does: Construct a multi-level video perturbation benchmark MLVP, investigate the robustness of video-text retrieval under query shift, and propose the HAT-VTR online adaptation framework.
π― What it does: Propose a hyperparameter-free gradient biasing method called DSD, which suppresses the collapse of SNNs under heterogeneous training and enhances robustness.
Robustness in Text-Attributed Graph Learning: Insights, Trade-offs, and New Defenses
Runlin Lei (Renmin University of China), Chuntao Hong (Ant Group)
CodeRepresentation LearningAdversarial AttackGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextGraph
π― What it does: A unified text-attribute graph (TAG) robustness evaluation framework is constructed, conducting systematic structural and text attack experiments on ten cross-domain datasets, and proposing an adaptive defense model based on LLM called SFT-auto.
ROC-n-reroll: How verifier imperfection affects test-time scaling
Florian E. Dorner (ETH Zurich), Fanny Yang (ETH Zurich)
CodeComputational EfficiencyTransformerLarge Language ModelTextBenchmark
π― What it does: This paper investigates the relationship between the performance and computational cost of two test-time scaling methods, rejection sampling (RS) and best-N (BoN), when using imperfect validators, through theoretical analysis and experimental validation, revealing that it depends on the geometric properties of the validator's ROC curve.
ROGA: Scaling Generalist Agents for Office Productivity Tasks via Tool Generation
Mugeng Liu (Peking University), Yun Ma (Peking University)
CodeTransformerLarge Language ModelAgentic AIWorld ModelTextTabularBenchmark
π― What it does: Proposes the ROGA framework to address the shortcomings of automated tool generation (ATG) agents in long-term, stateful office tasks, thereby improving task success rates.
Rote Learning Considered Useful: Generalizing over Memorized Data in LLMs
Qinyuan Wu (Max Planck Institute for Software Systems), Muhammad Bilal Zafar (Ruhr University Bochum)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Designed a two-phase 'rote learning-then-generalize' framework, first enabling LLMs to memorize fact triplets using semantically meaningless key tokens, then fine-tuning with extremely few semantic prompts to allow the model to generalize across various prompts, languages, and reasoning tasks
Routing Channel-Patch Dependencies in Time Series Forecasting with Graph Spectral Decomposition
Dongyuan Li (University of Tokyo), Renhe Jiang (University of Tokyo)
CodeGraph Neural NetworkMixture of ExpertsTime SeriesBenchmark
π― What it does: Propose the xCPD plugin, which leverages graph Fourier decomposition to model frequency domain dependencies in the channel-patch layer, enhancing the performance of multivariate time series prediction.
CodeGenerationComputational EfficiencyTransformerMixture of ExpertsDiffusion modelRectified FlowAuto EncoderContrastive LearningImageText
π― What it does: While expanding the Diffusion Transformer, the Mixture-of-Experts framework ProMoE is introduced, along with a two-step router and contrastive learning to enhance expert specialization.
RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation
Jane Luo (Microsoft), Mao Yang (Microsoft)
CodeAI Code AssistantTransformerLarge Language ModelTextGraphBenchmark
π― What it does: Designed and implemented the Repository Planning Graph (RPG) and ZeroRepo framework for generating complete software repositories from high-level requirements;
RPM: Reasoning-Level Personalization for Black-Box Large Language Models
Jieyong Kim (Yonsei University), Dongha Lee (Yonsei University)
CodeRecommendation SystemExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Proposed and implemented the RPM framework for reasoning-level personalization in black-box large language models, automatically extracting features from user history, clustering them into factors, constructing personalized reasoning paths, and retrieving matching examples through feature-based retrieval during inference;
π― What it does: Propose a new neural combinatorial optimization framework, RRNCO, specifically designed to address real-world complexities in vehicle routing problems, such as asymmetric distances, durations, and directional angles, and construct a VRP dataset containing asynchronous distance and duration matrices from 100 real cities.
π― What it does: Designed and implemented an offline-online reinforcement learning framework named SAC Flow, which can directly end-to-end train streaming policies and eliminate gradient explosion by viewing streaming policies as RNNs and reparameterizing them as GRU/Transformer.
SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training
Qi Zhang (Peking University), Yisen Wang (Peking University)
CodeDomain AdaptationExplainability and InterpretabilityRepresentation LearningLarge Language ModelPrompt EngineeringAuto EncoderContrastive LearningText
π― What it does: This paper proposes a method using sparse autoencoders (SAE) to predict the cross-domain transfer effectiveness of large models during post-training (e.g., supervised fine-tuning);
Safeguarding Multimodal Knowledge Copyright in the RAG-as-a-Service Environment
Tianyu Chen (ShanghaiTech University), Wenjie Wang (ShanghaiTech University)
CodeRetrievalSafty and PrivacyTransformerVision Language ModelDiffusion modelImageTextMultimodalityRetrieval-Augmented Generation
π― What it does: Proposes the AQUA framework for copyright watermarking of image knowledge in multimodal RAG systems, supporting both black-box and white-box usage.
SafeMoE: Safe Fine-Tuning for MoE LLMs by Aligning Harmful Input Routing
Jaehan Kim (KAIST), Sooel Son (KAIST)
CodeSafty and PrivacyLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
π― What it does: This paper proposes SAFEMOE, a safe fine-tuning method for Mixture-of-Experts (MoE) large language models, aimed at preventing safety degradation caused by routing drift during fine-tuning.
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning
Yiwei Chen (Michigan State University), Sijia Liu (Michigan State University)
CodeSafty and PrivacySupervised Fine-TuningVision Language ModelMultimodality
π― What it does: This paper investigates the 'safety hallucination' phenomenon in vision-language models (VLMs) during safe fine-tuning, where models form pseudo-correlations with text surface features through supervised learning, leading to vulnerability to word substitution attacks and excessive caution. Subsequently, the paper proposes using machine unlearning (NPO, RMU) methods to eliminate these pseudo-correlations without labeled data, enhancing model security.
Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study
Kaustubh Ponkshe (Mohamed bin Zayed University of Artificial Intelligence), Praneeth Vepakomma (Mohamed bin Zayed University of Artificial Intelligence)
CodeSafty and PrivacyTransformerSupervised Fine-TuningText
π― What it does: Empirical investigation on the safety alignment of large language models, examining whether safety behaviors can be linearly separated in weight or activation spaces.
π― What it does: This paper addresses the multi-view graph domain adaptation (MGDA) problem, proposing the SAGA framework, which achieves cross-domain node classification when the source graph has labels and the target graph lacks labels.
π― What it does: SAGE proposes a unified VPR training framework that leverages a frozen DINOv2 backbone and lightweight modules to construct an online-updatable geo-visual graph, and dynamically selects the most challenging samples for training through greedy weighted sampling.
Same Content, Different Representations: A Controlled Study for Table QA
Yue Zhang (University of Texas at Dallas), Nikita Bhutani (Megagon Labs)
CodeData SynthesisRepresentation LearningTransformerLarge Language ModelTabularBenchmark
π― What it does: Conduct control experiments on table QA, compare the effects of structured and semi-structured tables, and propose the REPAIRTQA diagnostic benchmark.
π― What it does: This paper investigates the sample complexity and representational capacity of the scaling paradigm during testing, proving that the sample complexity of self-consistency and optimal n-sampling can be separated, and proposes a generic Transformer architecture that achieves online learning through self-correction, enabling a single model to adaptively handle multi-task inference;
Sample Lottery: Unsupervised Discovery of Critical Instances for LLM Reasoning
Zhiping Xiao (University of Washington), Ming Zhang (Peking University)
CodeOptimizationData-Centric LearningTransformerLarge Language ModelReinforcement LearningText
π― What it does: This paper proposes an unsupervised framework named CONST for identifying critical samples in RLVR training, leveraging program volatility and result volatility combined with conformal prediction to select a small amount of high-value data for annotation and optimization.
Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
Xueyan Li (ETH Zurich), Jonas Geiping (ETH Zurich)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: Investigated shifting sampling strategies from exploration to 'correctness-priority' in reasoning tasks, proposing rank-wise calibrated truncation sampling methods such as Greedy-Threshold, Calibrated-TopK, and CalibratedΞ΅.
π― What it does: Designed a plugin that enhances the performance and efficiency of deep clustering through density-aware clustering head initialization and dynamic sample screening.
π― What it does: This paper proposes kernelized temporal difference (TD) estimators and KL-regularized natural gradient (NPG) policy improvement steps within the reproducing kernel Hilbert space (RKHS) framework, providing sampling complexity and non-asymptotic convergence rates for any RKHS (e.g., Sobolev, Gaussian, NTK, etc.).
π― What it does: Developed SAPIENS2, a series of high-resolution vision Transformers focused on human-centric visual tasks, with resolutions ranging from 1K to 4K, pre-trained and fine-tuned on a variety of dense prediction tasks using 1B portrait images.
SAVE: A Generalizable Framework for Multi-Condition Single-Cell Generation with Gene Block Attention
Jiahao Li (Fudan University), Fei Wang (Fudan University)
CodeGenerationData SynthesisTransformerLarge Language ModelFlow-based ModelAuto EncoderBiomedical Data
π― What it does: Propose a unified single-cell RNA-seq conditional generation framework SAVE, integrating Transformer gene block attention, VAE variational inference, and Flow Matching, which can achieve controllable generation of cell expression, batch correction, and perturbation prediction under various experimental conditions;
SC-Arena: A Natural Language Benchmark for Single-Cell Reasoning with Knowledge-Augmented Evaluation
Jiahao Zhao (Northeastern University), Min Yang (Northeastern University)
CodeGenerationLarge Language ModelBiomedical DataBenchmarkRetrieval-Augmented Generation
π― What it does: Propose the SC-ARENA benchmark, which designs five open-ended natural language tasks based on virtual cell abstraction (cell type annotation, cell description, cell generation, perturbation prediction, scientific question answering), and develops a knowledge-enhanced LLM-as-a-judge evaluation framework, providing interpretable, biology-knowledge-based scores and reasoning.
SCAD: Super-Class-Aware Debiasing for Long-Tailed Semi-Supervised Learning
Sunguk Jang (AITRICS), Byung-Jun Lee
CodeClassificationRepresentation LearningData-Centric LearningConvolutional Neural NetworkVision Language ModelImage
π― What it does: Proposes the Super-Class-Aware Debiasing (SCAD) framework to address the imbalance issue within the same super-class in long-tailed semi-supervised learning;
Scalable and Adaptive Trust-Region Learning via Projection Convex Hull
Hongyang Jia (Tsinghua University), Chongqing Kang (Tsinghua University)
CodeOptimizationExplainability and InterpretabilityTabularBiomedical Data
π― What it does: This paper proposes the Projection Convex Hull (PCH) framework for learning compact, interpretable polyhedral trust regions from labeled data.
π― What it does: Propose the Elastic Reasoning framework, which splits the reasoning process into a thinking phase and an answering phase, and employs budget constraints for truncation during inference.
π― What it does: Propose Dual Adversarial Training (DAT), replacing SGLD learning with PGD+BCE to achieve robust classification and high-quality generation simultaneously within a single framework.
π― What it does: Proposed a scalable reinforcement learning method called QFLEX, which performs targeted exploration in high-dimensional continuous control spaces through value-guided probabilistic flows.
Jinmei Liu (Nanjing University), Zhi Wang (Nanjing University)
CodeTransformerReinforcement LearningWorld Model
π― What it does: Proposed S-ICQL, a scalable context learning framework for offline multi-task reinforcement learning that combines dynamic programming with world models, using a multi-head Transformer to simultaneously predict policies and values;
π― What it does: Propose a scalable multi-task low-rank adaptation method called mtLoRA, which maintains or even improves model performance when the number of tasks increases significantly.
Scalable Multilingual Multimodal Machine Translation with Speech-Text Fusion
Yexing Du (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
CodeGenerationTransformerLarge Language ModelTextMultimodalityAudio
π― What it does: Propose a Speech-guided Machine Translation (SMT) framework that integrates text and synthesized speech, leveraging multimodal large language models (MLLM) to generate higher quality translations, and introduces a self-evolution mechanism to achieve unsupervised continuous improvement.
Scalable Training for Vector-Quantized Networks with 100% Codebook Utilization
Yifan Chang, Xingang Wang
CodeGenerationTransformerAuto EncoderImage
π― What it does: Proposed the VQBridge projector and FVQ training framework, achieving 100% codebook utilization in vector quantization networks and improving reconstruction and generation quality.
ScaleCap: Scalable Image Captioning via Dual-Modality Debiasing
Long Xing (Chinese University of Hong Kong), Dahua Lin (University of Science and Technology of China)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: Proposed a scalable image description method called ScaleCap, which continuously enriches and corrects image captions through iterative questioning and sentence scoring comparison.
ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data
Zhaoyang Liu (Hong Kong University of Science and Technology), Wenhai Wang (Hong Kong University of Science and Technology)
CodeData-Centric LearningAgentic AIVision Language ModelImageTextMultimodalitySequentialChain-of-Thought
π― What it does: Developed a cross-platform general-purpose computer usage proxy called ScaleCUA and built a dual-loop interactive data pipeline to collect large-scale GUI data across six major platforms (Windows, macOS, Linux, Android, iOS, Web);
Yan Xie (Xidian University), Yifei Wang (Amazon AGI SF Lab)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose sparse feature attention (SFA) and implement the FlashSFA kernel, enabling Transformers to maintain high-dimensional representation capabilities while reducing self-attention computation from O(n dΒ²) to O(n k / dΒ²), significantly lowering KV-cache and FLOPs.
Scaling Behavior of Discrete Diffusion Language Models
Dimitri von RΓΌtte, Antonio Orvieto (ETH Zurich)
CodeGenerationTransformerLarge Language ModelDiffusion modelText
π― What it does: Study and quantify the scaling behavior of discrete diffusion language models (DLMs) under different noise types (masking, uniform, hybrid), focusing on key hyperparameters such as batch size and learning rate;
CodeData SynthesisLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AITextTabular
π― What it does: Developed DATAMIND, a scalable data synthesis and training framework for building general-purpose data analysis agents, generating the DATAMIND-12K dataset, and subsequently training DATAMIND-7B/14B models;
Scaling Laws and Symmetry, Evidence from Neural Force Fields
Khang Ngo (Mila - Quebec AI Institute), Siamak Ravanbakhsh (Mila - Quebec AI Institute)
CodeDrug DiscoveryGraph Neural NetworkBiomedical DataPhysics Related
π― What it does: Conduct large-scale experiments on neural network models for molecular interatomic potential energy, systematically measuring the power-law scaling relationships in three dimensions (computational cost, data size, model scale) across different equivariance architectures (non-equivariant, low-order equivariant, high-order equivariant).
Scaling Multi-Task Bayesian Optimization with Large Language Models
Yimeng Zeng (University of Pennsylvania), Jacob R. Gardner (University of Pennsylvania)
CodeOptimizationHyperparameter SearchLarge Language ModelSupervised Fine-TuningTextBiomedical Data
π― What it does: In multi-task Bayesian optimization, the BOLT (Bayesian Optimization with LLM Transfer) strategy is proposed, leveraging large language models (LLMs) to provide high-quality candidate solutions only during the initialization phase, thereby accelerating optimization for new tasks;
Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
Zhaoyi Li (University of Science and Technology of China), Ying Wei (Zhejiang University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: Systematically analyze error patterns in LLMs during reasoning jump generalization, and propose a method to correct reasoning errors during testing by dynamically eliminating error-handling heads (TCR)
Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
Shengyin Sun (City University of Hong Kong), Chen Ma (Huawei Technologies)
CodeComputational EfficiencyTransformerLarge Language ModelTextBenchmark
π― What it does: Built and evaluated the first speculative decoding benchmark specifically designed for scaling scenarios during LLM testing (BoN and multi-round reasoning).
π― What it does: Proposed a no-training acceleration framework called ScalingCache for Diffusion Transformers, achieving extreme acceleration through differential scaling and dynamic caching.
CodeDrug DiscoveryTransformerFlow-based ModelBiomedical Data
π― What it does: scDFM proposes a distributed generative framework based on conditional flow matching and MMD regularization for predicting transcriptomes after single-cell perturbation.
Scheduling Your LLM Reinforcement Learning with Reasoning Trees
Hong Wang (Tencent), Jiawei Chen (Zhejiang University)
CodeLarge Language ModelReinforcement LearningText
π― What it does: Propose an r-score learning difficulty evaluation metric based on the reasoning tree structure, and design the Re-Schedule data scheduling algorithm to improve the inference performance of LLMs in RLVR.
Sci2Pol: Evaluating and Fine-tuning LLMs on Scientific-to-Policy Brief Generation
Weimin Wu (Northwestern University), Han Liu (Northwestern University)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkRetrieval-Augmented Generation
π― What it does: Proposed Sci2Pol-Bench and Sci2Pol-Corpus, which are evaluation benchmarks and training corpora for generating policy briefs from scientific papers, respectively, and conducted fine-grained evaluations of LLMs through a five-stage writing process.
SciNav: A General Agent Framework for Scientific Coding Tasks
TIANSHU ZHANG, Huan Sun (Ohio State University)
CodeAI Code AssistantLarge Language ModelAgentic AIText
π― What it does: Proposed SciNav, an autonomous agent framework for scientific programming tasks, which efficiently explores the solution space and generates executable code under limited search budgets by leveraging relative judgment-driven Top-K tree search (TKCTS).
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringMixture of ExpertsTime SeriesBenchmark
π― What it does: This paper constructs a scientific time series benchmark SciTS spanning 12 scientific fields, 7 task types, and 54,023 instances, performs zero-shot evaluation on 17 models (text LLMs, multimodal LLMs, unified time series models), and proposes a time series processing framework TimeOmni compatible with general-purpose LLMs.