Conference on Neural Information Processing Systems Β· 2283 papers
STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem
Hong Wang (University of Science and Technology of China), huanshuo dong
CodeOptimizationComputational EfficiencyPoint CloudPhysics Related
π― What it does: Proposes STNet, which solves the operator eigenvalue problem by performing spectral transformations on operators during the iterative process of neural networks.
π― What it does: A positive-positive learning method that does not require negative samples is proposed, utilizing noise to generate similar samples and achieving unsupervised feature learning through effective dimensionality reduction.
Stochastic Process Learning via Operator Flow Matching
Yaozhong Shi (California Institute of Technology), Kamyar Azizzadenesheli (NVIDIA Corporation)
CodeFlow-based ModelTime Series
π― What it does: The Operator Flow Matching (OFM) framework is proposed to learn the prior of stochastic processes in arbitrary domains and achieve analytical density and regression in function spaces.
Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning
Jie Cheng (Institute of Automation, Chinese Academy of Sciences), Fei-Yue Wang (Institute of Automation, Chinese Academy of Sciences)
CodeTransformerReinforcement LearningText
π― What it does: A reinforcement learning framework called PURE was designed and validated, utilizing a process reward model (PRM) to address the issue of reward hijacking that often occurs during PRM training.
π― What it does: The Straight-Line Diffusion Model (SLDM) is proposed, achieving efficient 3D molecular generation by designing a linear noise decay trajectory.
Strategic Classification with Non-Linear Classifiers
Benyamin Trachtenberg (Technion - Israel Institute of Technology), Nir Rosenfeld (Technion - Israel Institute of Technology)
CodeClassificationOptimizationTabular
π― What it does: This study investigates how nonlinear classifiers perform and their impact on learning in scenarios where users can modify features based on the classifier.
Strategic Cost Selection in Participatory Budgeting
Piotr Faliszewski (AGH University of Science and Technology), Mateusz Szwagierczak (AGH University of Science and Technology)
CodeOptimizationTabular
π― What it does: This study investigates how project proposers in participatory budgeting (PB) can achieve optimal benefits by setting project costs when they are aware of the voting outcomes, and analyzes whether pure Nash equilibria exist under different PB rules.
π― What it does: A dataset partitioning method for image segmentation is proposed, which includes pixel-level iterative partitioning (IPS) and genetic partitioning based on Wasserstein distance (WDES);
STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds
Yinfang Chen (University of Illinois Urbana-Champaign), Tianyin Xu (University of Illinois Urbana-Champaign)
CodeAnomaly DetectionOptimizationTransformerLarge Language ModelAgentic AITabularBenchmark
π― What it does: STRATUS has been constructed, a multi-agent system based on large language models, to achieve autonomous reliability engineering (SRE) for cloud services, including fault detection, localization, root cause analysis, and automatic mitigation; a complete transaction safety mechanism has been implemented on cloud platforms (such as Kubernetes);
StreamForest: Efficient Online Video Understanding with Persistent Event Memory
Xiangyu Zeng (Zhejiang University), Limin Wang (Noah's Ark Lab, Huawei)
CodeRecognitionAutonomous DrivingComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningVideoMultimodalityBenchmark
π― What it does: The StreamForest framework is proposed for real-time video understanding, consisting of two main modules: Fine-grained Spatiotemporal Window and Persistent Event Memory Forest, achieving efficient long-term memory and immediate perception.
Streaming Stochastic Submodular Maximization with On-Demand User Requests
Honglian Wang (KTH Royal Institute of Technology), Aristides Gionis (KTH Royal Institute of Technology)
CodeRecommendation SystemOptimizationTabular
π― What it does: This paper proposes a new streaming stochastic submodular maximization problemβS3MOR, which addresses the issue of users accessing and only being able to display k articles at once in the news recommendation scenario, and designs various low-memory algorithms.
STree: Speculative Tree Decoding for Hybrid State Space Models
Yangchao Wu (University of California Los Angeles), Stefano Soatto (University of California Los Angeles)
CodeTransformerLarge Language ModelText
π― What it does: This paper introduces STree, the first scalable tree-based speculative decoding algorithm for state space models (SSM) and their hybrid architecture with Transformers, enabling parallel generation of multi-step outputs and tree-based validation in a single forward pass.
STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimization
Diqi He (Northwestern Polytechnical University), Dingwen Zhang (Northwestern Polytechnical University)
CodeOptimizationRobotic IntelligenceTransformerLarge Language ModelVision Language ModelMultimodality
π― What it does: This paper proposes a zero-shot continuous visual and language navigation framework called STRIDER, which achieves efficient navigation in unseen 3D environments through structured path planning and task alignment adjustment.
CodeRecognitionObject DetectionTransformerLarge Language ModelPrompt EngineeringMultimodalityPoint CloudBenchmark
π― What it does: This paper proposes a structured 2D prompt framework, Struct2D, which utilizes BEV images, object tags, and metadata to enable multimodal large language models (MLLMs) to perform 3D spatial reasoning with only 2D inputs, and constructs a large-scale instruction tuning dataset, Struct2D-Set.
Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs
Yifan Wei (Beihang University), Li Du (Beijing Academy of Artificial Intelligence)
CodeData SynthesisOptimizationTransformerLarge Language ModelSupervised Fine-TuningAgentic AIPrompt EngineeringBiomedical Data
π― What it does: The SENATOR framework is proposed, which uses structure entropy-guided Monte Carlo tree search to locate knowledge gaps of LLMs on knowledge graphs and generate targeted synthetic data for self-repair.
π― What it does: This paper proposes SIHD, an adaptive multi-scale hierarchical diffusion framework that utilizes offline trajectory structural information to address long-term sparse reward tasks in offline reinforcement learning (RL).
Jianqiao Zheng (Australian Institute for Machine Learning), Simon Lucey (Australian Institute for Machine Learning)
CodeTransformerImage
π― What it does: This paper proposes embedding convolutional pulse filters into attention maps in Vision Transformer (ViT) solely through structured initialization to enhance performance on small-scale datasets while maintaining performance on large-scale datasets.
π― What it does: This paper studies an actor-critic structure embedded with a combinatorial optimization layer to address the reinforcement learning problem of combinatorial MDPs.
Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models
Aleksandar Terzic, Abbas Rahimi (IBM Research)
CodeTransformerTime SeriesSequential
π― What it does: A structurally sparse PD-SSM (PΓD) state space model is proposed, which can efficiently simulate any N-state finite automaton in a single-layer, N-dimensional state space.
StruDiCO: Structured Denoising Diffusion with Gradient-free Inference-stage Boosting for Memory and Time Efficient Combinatorial Optimization
Yu Wang (Jilin University), Yi Chang (Jilin University)
CodeOptimizationDiffusion modelGraph
π― What it does: A structured denoising diffusion framework, StruDiCO, is proposed to gradually construct interpretable combinatorial optimization solutions during the inference process.
π― What it does: This paper proposes a style imitation protection method called StyleGuard, based on latent space style perturbations, to prevent unauthorized style replication of artists' works by text-to-image diffusion models (such as DreamBooth and Textual Inversion).
π― What it does: Achieve voting-based ensemble by training multiple times on subsamples and taking the most frequently occurring model (or Ξ΅-optimal voting), thereby improving the model's generalization performance.
SubTrack++ : Gradient Subspace Tracking for Scalable LLM Training
Sahar Rajabi (University of Waterloo), Sirisha Rambhatla (University of Waterloo)
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The SubTrack++ method is proposed, which improves the memory and time efficiency of LLM training by tracking gradient subspaces on the Grassmannian and combining it with a projection-aware optimizer;
SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications
Gabriele Oliaro (Carnegie Mellon University), Aurick Qiao (Snowflake AI Research)
CodeComputational EfficiencyLarge Language ModelAgentic AITextBenchmark
π― What it does: A model-free speculative decoding method based on suffix trees, called SuffixDecoding, is proposed to accelerate inference in proxy-based AI applications.
Sum Estimation under Personalized Local Differential Privacy
Dajun Sun (Hong Kong University of Science and Technology), Graham Cormode (University of Warwick)
CodeSafty and PrivacyGaussian SplattingImageTabular
π― What it does: The research addresses the problem of sum/mean estimation under personalized local differential privacy and proposes two new protocols.
Yizhou Liu (Massachusetts Institute of Technology), Jeff Gore (Massachusetts Institute of Technology)
CodeTransformerLarge Language ModelAuto EncoderText
π― What it does: This study investigates the impact of representation superposition on loss scaling in large models, proposing two scenarios: weak superposition and strong superposition, and provides corresponding loss scaling laws.
SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution
Yuxiang Wei (University of Illinois Urbana-Champaign), Sida Wang
CodeTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: This paper proposes the SWE-RL method, which utilizes rule-based rewards and real open-source software evolution data (PR) to perform reinforcement learning on large language models, thereby enhancing their reasoning and repair capabilities in the field of software engineering, particularly in solving real GitHub issues.
SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
Xiao Liang (University of California), Weizhu Chen (Microsoft)
CodeLarge Language ModelReinforcement LearningText
π― What it does: An adaptive weakness-driven problem synthesis framework, SwS, is proposed. In the training of Reinforcement Learning-based Visual Reasoning (RLVR), weaknesses are automatically identified based on the model's continuous failures during the pre-training phase, and targeted new problems are generated accordingly. These synthesized problems are then added to the training set for reinforcement learning.
π― What it does: The SymMaP framework is proposed to learn efficient preprocessing parameter expressions through symbolic discovery, enhancing the efficiency of solving linear systems.
SYMPHONY: Synergistic Multi-agent Planning with Heterogeneous Language Model Assembly
Wei Zhu (Yunnan University), Kun Yue (Yunnan University)
CodeTransformerLarge Language ModelAgentic AIText
π― What it does: A multi-agent planning framework named SYMPHONY is proposed, which integrates various large language models (LLMs) into Monte Carlo Tree Search (MCTS) to enhance search diversity and planning efficiency through model heterogeneity.
SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning
Yiting Wang (University of Maryland), Ang Li (University of Maryland)
CodeOptimizationAI Code AssistantTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Proposes the SymRTLO framework, which combines large language models with symbolic reasoning to achieve automatic rewriting and optimization of RTL code.
π― What it does: The SynBrain framework is proposed, which models the mapping from vision to fMRI using probability distributions, simulating one-to-many neural variations while maintaining functional consistency, and supporting few-shot cross-subject adaptation.
π― What it does: To address the challenge of joint training for detection and tracking in multi-camera 3D multi-object tracking, a SynCL training strategy is proposed.
Mengshi Qi (Beijing University of Posts and Telecommunications), Huadong Ma (Beijing University of Posts and Telecommunications)
CodeTransformerLarge Language ModelVision Language ModelTextMultimodality
π― What it does: A collaborative scheduling scheme is proposed to simultaneously reduce the communication bottleneck of tensor parallelism (TP) and the synchronization bottleneck of pipeline parallelism (PP). It interleaves fine-grained computation blocks to form woven execution blocks, thereby achieving efficient overlap between TP communication and PP computation.
SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond
Junteng Liu (Hong Kong University of Science and Technology), Junxian He (Hong Kong University of Science and Technology)
CodeData SynthesisReinforcement LearningText
π― What it does: A scalable logical reasoning data synthesis framework, SYNLOGIC, has been constructed, generating verifiable reasoning data with 35 tasks and adjustable difficulty; reinforcement learning (RL) training has been conducted based on this data; furthermore, mixing logical data with mathematical and coding data has improved multi-task learning efficiency; state-of-the-art performance has been achieved on multiple logical and mathematical benchmarks.
Synthesize Privacy-Preserving High-Resolution Images via Private Textual Intermediaries
Haoxiang Wang (Peking University), Huishuai Zhang (Peking University)
CodeGenerationData SynthesisSafty and PrivacyTransformerLarge Language ModelDiffusion modelImageTextMultimodality
π― What it does: By first converting private images into textual descriptions, then using a private evolutionary algorithm to generate differentially private sentences in the text domain, and finally reconstructing these sentences into high-resolution synthetic images using a text-to-image diffusion model, this approach achieves differential privacy image generation without training costs.
Synthesizing Performance Constraints for Evaluating and Improving Code Efficiency
Jun Yang (University of Chicago), Kexin Pei (University of Chicago)
CodeOptimizationComputational EfficiencyLarge Language ModelTextChain-of-Thought
π― What it does: This paper proposes the WEDGE framework, which utilizes LLM to synthesize performance constraints and combines coverage-guided fuzz testing to automatically generate efficient stress tests for code implementations.
π― What it does: This paper proposes a method for infinitely generating time-series-symbol (SΒ²) data pairs and based on this, the SymTime temporal pre-training model trained on large-scale synthetic data.
π― What it does: This paper proposes Synthetic-Powered Predictive Inference (SPI), a framework that utilizes synthetic data to enhance the efficiency of synthetic calibration samples while maintaining distribution-independent coverage guarantees.
Yumin Choi (Korea Advanced Institute of Science and Technology), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
CodeOptimizationMeta LearningTransformerLarge Language ModelPrompt EngineeringText
π― What it does: A bilevel system prompt optimization framework is proposed and implemented, utilizing meta-learning to automatically optimize system prompts across multiple tasks and domains, enhancing the generalization and adaptation performance of large language models on unknown tasks and user prompts.
π― What it does: A system-embedded diffusion bridge model (SDB) is proposed, which achieves a generative solution to the inverse problem by directly embedding the linear measurement system into the coefficients of matrix-valued stochastic differential equations;
T-norm Selection for Object Detection in Autonomous Driving with Logical Constraints
Thomas Eiter (Vienna University of Technology), Sota Moriyama (National Institute of Informatics)
CodeObject DetectionAutonomous DrivingImage
π― What it does: This paper proposes a neural-symbolic framework MOD-ECL, which integrates logical constraints into multi-label object detection in autonomous driving using t-norms, and adaptively selects the optimal t-norm and dynamically adjusts the regularization coefficient Ξ» of the constraint loss during the training process.
T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning
Yanjun Fu (University of Maryland), Sanghamitra Dutta (University of Maryland)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A data selection framework for instruction fine-tuning called T-SHIRT is proposed, which combines token-level information with sample neighborhood robustness assessment.
π― What it does: This paper proposes a text-to-image generation model T2I-R1 based on dual-layer chain-of-thought (semantic-level CoT and token-level CoT), and achieves joint optimization of the two levels of thinking through reinforcement learning.
TabDPT: Scaling Tabular Foundation Models on Real Data
Junwei Ma (Layer 6 AI), Maksims Volkovs (Layer 6 AI)
CodeClassificationTransformerTabular
π― What it does: This paper presents TabDPT, a table-based model based on row-level Transformers, which utilizes retrieval-based context and self-supervised learning for pre-training, enabling direct inference on unseen classification and regression tasks.
Liyao Li (Zhejiang University), Junbo Zhao (Zhejiang University)
CodeGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringMultimodalityTabularBenchmark
π― What it does: The TAMO framework is proposed, which integrates tables as an independent modality with large language models and retains the structural information of tables through a hypergraph encoder; simultaneously, a StructQA benchmark dataset focused on table structure understanding is constructed.
Table2LaTeX-RL: High-Fidelity LaTeX Code Generation from Table Images via Reinforced Multimodal Language Models
Jun Ling (University of Electronic Science and Technology of China), Peng Wang (University of Electronic Science and Technology of China)
CodeGenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTabular
π― What it does: This study addresses the task of generating LaTeX code from table images, aiming to automatically reconstruct high-quality, publication-ready tables.
TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields
Alan Arazi (Technion - Israel Institute of Technology), Roi Reichart
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningTabular
π― What it does: A foundational model called TabSTAR is proposed and implemented, which is oriented towards table text features, supports cross-dataset transfer learning, and can quickly adapt to downstream tasks after multi-task pre-training.
CodeFederated LearningSafty and PrivacyTransformerContrastive LearningTabularBiomedical Data
π― What it does: TABULA, a self-supervised tabular pre-training model for single-cell transcriptomics, has been developed, achieving privacy-preserving pre-training and fine-tuning for downstream tasks such as cell type annotation, gene imputation, and gene perturbation prediction within a federated learning framework.
π― What it does: This paper proposes FedPFT, a method that addresses the feature-classifier mismatch problem during the federated learning training process through personalized prompts and a global self-attention feature transformation module, thereby achieving a better personalized model.
π― What it does: TADA is proposed, a training-independent diffusion sampling method that achieves fast sampling using higher-dimensional initial noise and momentum dynamics.
TAMI: Taming Heterogeneity in Temporal Interactions for Temporal Graph Link Prediction
Zhongyi Yu (Beijing Normal-Hong Kong Baptist University), Weipeng Zhuo (Beijing Normal-Hong Kong Baptist University)
CodeGraph Neural NetworkGraphTime SeriesFinance Related
π― What it does: This study proposes the TAMI framework, aimed at addressing the negative impact of node interaction heterogeneity in continuous time graphs on link prediction.
π― What it does: The TANDEM framework is proposed, utilizing dual network dynamic synchronization to solve the dual-layer optimization problem of data mixing ratios.
π― What it does: A one-shot generative time series forecasting framework named TARFVAE is proposed, which enhances posterior estimation capabilities by combining Transformer-based Autoregressive Flow (TARFLOW) with VAE, achieving high-quality deterministic and uncertainty predictions while maintaining one-shot inference speed.
CodeConvolutional Neural NetworkRecurrent Neural NetworkTransformerAuto EncoderContrastive LearningTime SeriesSequentialPhysics Related
π― What it does: Modeling the task optimization of tactile perception in mice using time neural networks, exploring the performance of convolutional recurrent networks and self-supervised learning in matching cortical neural responses.
π― What it does: A method based on sparse autoencoders called Monosemantic Neural Activation (MONA) is proposed for task-specific data selection in instruction tuning.
π― What it does: This paper proposes a conditional backdoor attack method for knowledge distillation, which activates the backdoor of the teacher model in the student model.
Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignment
Weixiang Zhao (Harbin Institute of Technology), Ting Liu (Harbin Institute of Technology)
CodeTransformerLarge Language ModelReinforcement LearningText
π― What it does: Proposes the RLPA framework, which utilizes reinforcement learning to dynamically construct and update user profiles in multi-turn dialogues, achieving personalized alignment.
Chengpeng Li (University of Science and Technology of China), Dayiheng Liu
CodeAI Code AssistantReinforcement Learning from Human FeedbackLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
π― What it does: This paper proposes the CoRT framework, which efficiently utilizes a code interpreter in large reasoning models (LRM) for mathematical reasoning, and achieves the collaboration of code and internal reasoning through prompt engineering, rejection fine-tuning, and reinforcement learning.
π― What it does: By allowing the Transformer to perform trial-and-error searches in solving NP-class combinatorial problems like Sudoku, it learns rule application, guessing, and backtracking, achieving nearly perfect problem-solving capabilities.
Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context
Taejong Joo (Northwestern University), Diego Klabjan (Northwestern University)
CodeComputational EfficiencyMeta LearningTransformerLarge Language ModelText
π― What it does: This paper proposes an evaluation method based on the Meta ICL framework, systematically comparing the non-parametric learning capabilities of large language models with the sample complexity of Bayesian optimal and other benchmark algorithms. It finds that although the performance is close to optimal with few samples, the efficiency of ICL significantly decreases as the number of examples increases.
π― What it does: An end-to-end rate-distortion (RD) optimization compression framework oriented towards 4D Gaussian expansion (4DGS) is proposed, enabling dynamic scene rendering with flexible control over compression quality and bit rate.
Luckeciano Carvalho Melo, Yarin Gal (University of Oxford)
CodeReinforcement LearningImage
π― What it does: A new variational continual learning objective is proposed - Temporal-Difference VCL (TD-VCL), which introduces multi-step (n-step) KL regularization and the TD(Ξ») mechanism into the learning objective, utilizing past multiple posterior estimates to alleviate catastrophic forgetting.
TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs
Yunheng Li (Nankai University), Ming-Ming Cheng (Nankai University)
CodeLarge Language ModelReinforcement LearningVideoMultimodalityChain-of-Thought
π― What it does: The TempSamp-R1 framework is proposed, which enhances the performance of multimodal large models in video temporal localization tasks through reinforcement learning combined with offline supervision.
π― What it does: The paper proposes Tensor Decomposition Networks (TDN), which accelerates molecular potential energy calculations by replacing SO(3) equivariant tensor products with low-rank CANDECOMP/PARAFAC (CP) decomposition.
π― What it does: Proposes the Tensor Product Attention (TPA) mechanism, which significantly compresses the KV cache by performing context-aware low-rank tensor decomposition on queries, keys, and values; builds the T6 Transformer based on TPA and implements the FlashTPA decoder;
TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search
Akash Kundu (University of Helsinki), Stefano Mangini (University of Helsinki)
CodeOptimizationNeural Architecture SearchReinforcement LearningTabularPhysics Related
π― What it does: Combining the ground state (MPS) pre-trained with tensor networks and reinforcement learning to automatically search for and optimize quantum circuit architectures suitable for NISQ hardware.
Yingnan Liu (National University of Singapore), Wynne Hsu (National University of Singapore)
CodeDomain AdaptationImage
π― What it does: A gradient-free test-time adaptation method called TACT is proposed, which enhances the model's robustness under distribution shifts by trimming non-causal features.
CodeSegmentationDomain AdaptationTransformerPrompt EngineeringVision Language ModelImageMultimodality
π― What it does: This paper proposes a testing-time adaptation framework called MLMP for open-vocabulary semantic segmentation (OVSS), which dynamically adjusts the model during inference to cope with domain shifts.
Test-Time Adaptive Object Detection with Foundation Model
Yingjie Gao (Beihang University), Di Huang (Beihang University)
CodeObject DetectionDomain AdaptationTransformerPrompt EngineeringVision Language ModelImageMultimodality
π― What it does: The paper proposes a test-time adaptive object detection method based on a visual-language foundation model (Grounding DINO), which can adapt in real-time without accessing source data, across domains, and even across categories.
Test-Time Spectrum-Aware Latent Steering for Zero-Shot Generalization in Vision-Language Models
Konstantinos M. Dafnis (Rutgers University), Dimitris N. Metaxas (Rutgers University)
CodeDomain AdaptationOptimizationComputational EfficiencyTransformerVision Language ModelContrastive LearningImage
π― What it does: A lightweight framework called STS is proposed for adaptive testing on visual-language models (such as CLIP). This framework learns a small coefficient vector in a low-dimensional spectral subspace obtained from the singular value decomposition (SVD) of text prototypes, which offsets the original text prototypes, thereby enhancing the robustness of zero-shot inference without modifying the frozen encoder.
Yuheng Yuan (National University of Singapore), Xinchao Wang (National University of Singapore)
CodeDepth EstimationImage
π― What it does: A Test3R method is proposed for learning 3D reconstruction by maximizing the point cloud consistency of different image pairs during the testing phase.
Tengjie Li (Shanghai Jiao Tong University), Lei Xu (Guangdong Laboratory of Artificial Intelligence and Digital Economy)
CodeGenerationDiffusion modelImage
π― What it does: A training-free multi-style sketch generation framework M3S is proposed, utilizing a diffusion model to achieve zero-shot style control by combining reference style sketches and text prompts.
Text-to-Code Generation for Modular Building Layouts in Building Information Modeling
Yinyi WEI, Xiao LI
CodeGenerationData SynthesisAI Code AssistantLarge Language ModelSupervised Fine-TuningText
π― What it does: A framework named Text2MBL is proposed for generating BIM code from text, which can directly convert natural language descriptions of modular building layouts into executable BIM code, automatically constructing the hierarchical structure of modules, units, and rooms.
π― What it does: This study explores how to utilize natural language supervision to train a general agent for offline meta reinforcement learning, achieving zero-shot text-to-decision generation.
π― What it does: This study investigates the role of attention heads in retrieval-augmented language models, distinguishing and locating in-context heads responsible for instruction understanding and information retrieval in in-context learning (ICL) from parametric heads that store relational knowledge. It demonstrates their causal impact on answer generation through functional vectors and weight modifications, and further develops a linear detector to track the sources of retrieved information.
The Computational Advantage of Depth in Learning High-Dimensional Hierarchical Targets
Yatin Dandi (Ecole Polytechnique Federale de Lausanne), Florent Krzakala (Ecole Polytechnique Federale de Lausanne)
CodeOptimizationComputational EfficiencyTabular
π― What it does: This paper characterizes the hierarchical feature learning process of deep networks by constructing single-index high-dimensional hierarchical objectives (SIGHT) and multi-index hierarchical objectives (MIGHT). It analyzes the learning dynamics of gradient descent training in the high-dimensional limit, proving that deep networks achieve lower sample complexity through layer-wise dimensionality reduction. It also provides a strict theorem for three-layer networks and proposes a general hierarchical information index (CIE) hypothesis.
The Emergence of Abstract Thought in Large Language Models Beyond Any Language
Yuxin Chen (National University of Singapore), Wenxuan Zhang (Singapore University of Technology and Design)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper analyzes the neuron activations of large language models to identify and distinguish language-related shared and exclusive neurons, studying their evolution with model iterations, and proposes a neuron-directed training method based on language independence scoring to enhance multilingual capabilities.
π― What it does: This paper proposes Flood complex, a filtering-style simplicial complex constructed on subsampling points, which can efficiently compute persistent homology (PH) on millions of point clouds.
The Fragile Truth of Saliency: Improving LLM Input Attribution via Attention Bias Optimization
Yihua Zhang (Michigan State University), Sijia Liu (Michigan State University)
CodeOptimizationExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
π― What it does: A new input saliency method called ABO (Attention Bias Optimization) is proposed, along with a long text saliency assessment framework based on needle-in-a-haystack (NIAH).
π― What it does: The first semantic-level image watermark removal attack is proposed - Next Frame Prediction Attack (NFPA), which transforms the watermark removal task into the next frame prediction of video generation, achieving efficient removal of eight existing SOTA watermark schemes.
π― What it does: Proposed the 'Indra Representation' - a relational representation method based on category theory, which constructs a global relationship vector for each sample using the relative distances of features generated by the model, and validates its effectiveness in unimodal, vision-language, and speech-language cross-modal tasks.
The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement
Ruihan Yang (Fudan University), Deqing Yang (Fudan University)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningAgentic AIText
π― What it does: Proposed and implemented Critique-Guided Improvement (CGI), a two-player framework that separates the actor (generating candidate actions) from the critic (providing natural language critiques and correction suggestions), utilizing critique to guide LLM agents in continuously improving their decisions in interactive environments.
π― What it does: Developed The Matrix, a world simulator capable of generating unlimited 720p high-fidelity video streams with real-time frame-level interactive control at 16 FPS, and can transfer to real-world environments under zero-shot conditions.
The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs?
Hao Yin (University of Science and Technology of China), Zilei Wang (University of Science and Technology of China)
CodeLarge Language ModelPrompt EngineeringContrastive LearningMultimodality
π― What it does: Evaluate and reveal the limitations of contrastive decoding methods in alleviating hallucinations in multimodal large language models, demonstrating that the improvement mainly stems from unidirectional adjustments to the output distribution and adaptive realizable constraints;
The Overthinker's DIET: Cutting Token Calories with DIfficulty-AwarE Training
Weize Chen (Tsinghua University), Maosong Sun (Tsinghua University)
CodeCompressionTransformerLarge Language ModelReinforcement LearningText
π― What it does: Proposes the DIET framework, which utilizes RL combined with real-time difficulty estimation to dynamically compress tokens generated by LLMs, significantly reducing verbose outputs caused by excessive reasoning;
The Promise of RL for Autoregressive Image Editing
Saba Ahmadi (Mila - Quebec AI Institute), Aishwarya Agrawal (Mila - Quebec AI Institute)
CodeImage TranslationGenerationOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningImageMultimodalityChain-of-Thought
π― What it does: A self-regressive model-based image editing framework called EARL is proposed, utilizing three training paradigms: supervised fine-tuning, reinforcement learning, and chain of thought (CoT). It ultimately proves that RL combined with a multimodal LLM discriminator is the most effective strategy.
π― What it does: A graph-level autoencoder GRALE is proposed, which can encode graphs of arbitrary size into a shared Euclidean space and decode back to the complete graph from this embedding space.
π― What it does: This study investigates the structure of depth convolution kernels in depth separable convolution networks, proposing and validating the hypothesis that only 8 principal key filters are needed to approximate the original thousands of convolution kernels.
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: This study investigates the impact of positive and negative rewards in RLVR on the reasoning performance of large language models, finding that using only negative sample reinforcement can significantly enhance reasoning effectiveness.
π― What it does: By transforming graphs into visual images and utilizing existing visual models to learn graph structures, a new benchmark called GraphAbstract is proposed to evaluate models' cognitive abilities regarding global graph structures.
The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
Shivam Agarwal (University of Illinois Urbana Champaign), Hao Peng (University of Illinois Urbana Champaign)
CodeOptimizationTransformerLarge Language ModelReinforcement LearningTextPhysics Related
π― What it does: Three unsupervised post-training and inference methods based on entropy minimization (EM-FT, EM-RL, EM-INF) are proposed and evaluated, enhancing the performance of large language models on complex reasoning, physics, and programming tasks by using only unlabeled data or without updating model parameters.
Theoretical Guarantees for the Retention of Strict Nash Equilibria by Coevolutionary Algorithms
Alistair Benford (University of Birmingham), Per Kristian Lehre (University of Birmingham)
CodeOptimization
π― What it does: This paper studies the stability of co-evolutionary algorithms (CoEAs) in maintaining strict Nash equilibrium under multiple action spaces, providing theoretical thresholds regarding mutation strength, selection operations, and stability. It also validates the theoretical limits through empirical experiments and further derives the upper bound of regret for CoEAs based on the stability results.
Think before Recommendation: Autonomous Reasoning-enhanced Recommender
Xiaoyu Kong (Taobao and Tmall Group of Alibaba), Xiang Wang (National University of Singapore)
CodeRecommendation SystemTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
π― What it does: Proposed two reinforcement learning-based LLM recommendation frameworks, RecZero and RecOne, which learn reasoning and scoring predictions directly on a single LLM without the need for a teacher model and multi-stage distillation.