Conference on Neural Information Processing Systems · 2283 papers
Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging
Ibrahim Ethem Hamamci (University of Zurich), Bjoern Menze (University of Zurich)
CodeGenerationData SynthesisConvolutional Neural NetworkTransformerVision Language ModelDiffusion modelImageTextBiomedical DataComputed Tomography
🎯 What it does: The BTB3D framework is proposed, utilizing a causal convolution encoder-decoder to generate fine 3D medical image labels, achieving new performance breakthroughs in CT report generation and text-to-CT synthesis tasks.
CodeCompressionComputational EfficiencyTransformerLarge Language ModelVision Language ModelImageVideoMultimodality
🎯 What it does: A training-free, model-agnostic visual token pruning method called CDPruner is proposed, which significantly reduces inference costs while maintaining the performance of multimodal large language models.
Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
Areeb Ahmad (Indian Institute of Technology Kanpur), Ashutosh Modi (Indian Institute of Technology Kanpur)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Decomposes the attention heads and MLP layers within the Transformer into orthogonal singular vector directions to achieve fine-grained explanations of the model's internal computations; and selectively adjusts these directions using learned diagonal masks.
Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliability
Divya Jyoti Bajpai (Indian Institute of Technology Bombay), Manjesh Kumar Hanawal (Indian Institute of Technology Bombay)
CodeClassificationComputational EfficiencyText
🎯 What it does: This study focuses on risk control in Early-Exit deep networks, proposing the UAT framework that uses a multi-armed bandit dynamic adaptive threshold to balance speed and accuracy during inference.
Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
Shiwei Li (Huazhong University of Science and Technology), Ruixuan Li (Huazhong University of Science and Technology)
CodeTransformerSupervised Fine-TuningText
🎯 What it does: Proposes Token-wise Projected Low-Rank Adaptation (TopLoRA), a parameter-efficient fine-tuning method that dynamically generates projection matrices for each input token while maintaining low rank.
🎯 What it does: A new multimodal dataset distillation framework called RepBlend is proposed, which compresses large image-text datasets into compact datasets while maintaining cross-modal learning effectiveness, addressing the modal collapse issue present in existing methods.
Beyond Oracle: Verifier-Supervision for Instruction Hierarchy in Reasoning and Instruction-Tuned LLMs
Sian-Yao Huang (CyCraft AI Lab), Cheng-Lin Yang (CyCraft AI Lab)
CodeLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: A unified instruction-level framework has been designed and implemented, utilizing an executable checker to supervise synthesized conflicting instruction instances, thereby achieving multi-level instruction alignment without the need for oracle labels.
🎯 What it does: A global constraint-based multi-resolution functional brain network learning framework GCM is proposed and implemented, which can directly mine high-order dependencies from multivariate time series and generate discrete functional brain networks.
🎯 What it does: By using backward discretization and learning proximal operators, we propose Proximal Diffusion Models (ProxDM) as an alternative to traditional score-based diffusion sampling methods.
Beyond Single-Task: Robust Multi-Task Length Generalization for LLMs
Yi Hu (Institute for Artificial Intelligence, Peking University), Muhan Zhang (Institute for Artificial Intelligence, Peking University)
CodeMeta LearningTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: A multi-task post-training framework called Meta-RFFT is proposed to enhance the length generalization ability of large language models on unseen tasks.
Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
Peng Lai (Southern University of Science and Technology), Guanhua Chen (Tsinghua University)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: The LAGER framework is proposed, which enhances the consistency between LLM-as-a-Judge and human ratings by aggregating the logits from multiple hidden layers and calculating the expected score, without altering the backbone of the LLM.
Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions
Liuyuan Jiang (University of Rochester), Tianyi Chen (Cornell University)
CodeOptimizationLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A new penalty-based bi-level optimization algorithm without value function evaluation (PBGD-Free) is proposed, which can be directly used for fine-tuning large-scale language model parameters.
Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models
Tyler A. Chang (University of California San Diego), Ben Bergen
CodeTransformerLarge Language ModelText
🎯 What it does: Identify and train the minimal subnetworks (bigram subnetworks) in Transformer language models that can predict the next word using only the current word, and study their structure and role in the residual flow.
🎯 What it does: This paper proposes a unified bi-level optimization framework that reformulates traditional min-max problems in adversarial learning (such as SAM and GAN) into a solvable lower-level problem, and designs a single-loop stochastic gradient algorithm for implementation.
Yuning Cui (Sun Yat-sen University), Alois Knoll (Technical University of Munich)
CodeRestorationTransformerImage
🎯 What it does: This paper proposes an efficient and general image restoration framework called BioIR, which utilizes two bionic modules—Peripheral to Fovea (P2F) and Fovea to Peripheral (F2P)—to achieve bidirectional interaction between local details and global context, thereby enabling high-quality image recovery in various scenarios such as single denoising, universal restoration, and composite degradation.
BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model
Adibvafa Fallahpour (University of Toronto), BO WANG
CodeDrug DiscoveryTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningMultimodalityBiomedical Data
🎯 What it does: BIOREASON has been developed, a multimodal framework that integrates DNA foundational models with large language models, capable of performing multi-step biological reasoning and variant effect prediction on genomic sequences.
BIPNN: Learning to Solve Binary Integer Programming via Hypergraph Neural Networks
Sen Bai (Changchun University of Science and Technology), Zhengang Jiang (Changchun University of Science and Technology)
CodeOptimizationGraph Neural NetworkGraph
🎯 What it does: This paper proposes an unsupervised binary integer programming solving framework based on hypergraph neural networks, called BIPNN, which transforms nonlinear BIP into polynomial unconstrained optimization (PUBO) and trains the hypergraph CNN using this as a loss.
Louis Kerner (CISPA Helmholtz Center for Information Security), Adam Dziedzic (CISPA Helmholtz Center for Information Security)
CodeGenerationData SynthesisImage
🎯 What it does: This paper proposes BitMark, a watermark that can be embedded at the bit level and is detectable and radiative. It subtly shifts the generated bit sequence during the autoregressive generation process of images, allowing for the identification of model-generated content while maintaining image quality and generation speed.
Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing
Jinhua Yin (Tsinghua University), Tao Qi (Beijing University of Posts and Telecommunications)
CodeAdversarial AttackTransformerVision Language ModelImageTextMultimodality
🎯 What it does: A black-box membership inference attack framework (KCMP) for large visual-language models is proposed and implemented, which can determine whether a training sample is included by only using the text output generated by the model.
🎯 What it does: A 'black-box source tracing' method based on language model memory is proposed to detect whether another model or text is based on its own training run;
Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers
Kazuki Irie (Harvard University), Samuel J. Gershman (Harvard University)
CodeRetrievalTransformerLarge Language ModelReinforcement LearningTextSequential
🎯 What it does: Three types of Hybrid Quadratic-Linear Transformers are proposed and evaluated, which achieve a sequence model with long context, precise retrieval, and expressive power by integrating KV attention and DeltaNet's fast weight programming.
Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
Xinyu Wang (University of Warwick), Matthäus Kleindessner (Amazon Web Services)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposes the Block-Diagonal LoRA (BD-LoRA) method to eliminate the communication overhead of S-LoRA in multi-device inference with multiple LoRA adapters;
BlockScan: Detecting Anomalies in Blockchain Transactions
Jiahao Yu (University of California Santa Barbara), Xinyu Xing (Northwestern University)
CodeAnomaly DetectionTransformerLarge Language ModelTabularTime Series
🎯 What it does: Designed and implemented BlockScan, a Transformer-based framework for detecting anomalies in blockchain transactions, capable of identifying malicious transactions in the DeFi ecosystem.
🎯 What it does: A Blockwise Flow Matching (BFM) framework is designed and implemented, dividing the generation process into several time segments, with each segment modeled using small dedicated velocity blocks, and improving generation quality and inference efficiency through semantic feature guidance and residual feature approximation techniques.
BlurDM: A Blur Diffusion Model for Image Deblurring
Jin-Ting He (National Yang Ming Chiao Tung University), Yen-Yu Lin (National Tsing Hua University)
CodeRestorationDiffusion modelImage
🎯 What it does: A diffusion model that integrates the fuzzy formation process (BlurDM) is proposed for single image deblurring, incorporating it as a prior into the latent space of existing deblurring networks.
BlurGuard: A Simple Approach for Robustifying Image Protection Against AI-Powered Editing
Jinsu Kim (Korea University), Jongheon Jeong (Korea University)
CodeAdversarial AttackDiffusion modelImage
🎯 What it does: A method for generating adversarial noise through adaptive Gaussian blur and spectral regularization is proposed to enhance the robustness of images under text-to-image model editing.
🎯 What it does: A bidirectional memory pool rewriting (BMW) mechanism is proposed, which updates using gradient descent while considering intra-class compactness and inter-class separation, improving memory pool updates in unsupervised person re-identification.
BoltzNCE: Learning likelihoods for Boltzmann Generation with Stochastic Interpolants and Noise Contrastive Estimation
Rishal Aggarwal (University of Pittsburgh), David Koes
CodeGenerationData SynthesisDrug DiscoveryScore-based ModelFlow-based ModelContrastive LearningSequentialBiomedical Data
🎯 What it does: A model called BoltzNCE is proposed to accelerate the Boltzmann generator by learning the likelihood of an energy-based model to approximate a flow generator, enabling fast sampling.
Boosting Knowledge Utilization in Multimodal Large Language Models via Adaptive Logits Fusion and Attention Reallocation
Wenbin An (Xi'an Jiaotong University), Shijian Lu (Nanyang Technological University)
CodeGenerationRetrievalTransformerLarge Language ModelTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: ALFAR is proposed, a plugin method that does not require additional training to enhance the knowledge utilization efficiency of multimodal large language models in retrieval-augmented generation (MRAG).
Guojian Zhan (Tsinghua University), Shengbo Eben Li (Tsinghua University)
CodeReinforcement LearningWorld ModelSequential
🎯 What it does: A framework named BOOM is proposed, which combines online planning and offline learning through a bootstrap loop between the planner and the policy, eliminating actor-bias;
Bootstrapping Hierarchical Autoregressive Formal Reasoner with Chain-of-Proxy-Autoformalization
Qi Liu (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
CodeLarge Language ModelText
🎯 What it does: This paper proposes a hierarchical autoregressive formal reasoner HAR and a chain-based agent automatic formalization CoPA to address the issues of mismatched step granularity and data scarcity in formal problem solving.
Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities
Mayank Jobanputra (Saarland University), Michael Hahn (Saarland University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: By constructing a series of retrieval and replication tasks, combined with a theoretical framework (C-RASP[pos]) and experiments, this study investigates the performance of large-scale pre-trained Transformers in terms of length generalization, directional bias, and uniqueness bias, and explores how fine-tuning can eliminate directional bias.
🎯 What it does: The first multimodal brain base model, Brain Harmony (BrainHarmonix), has been constructed to unify brain structural morphology and functional dynamics into a 1D marker.
🎯 What it does: Using bilingual brain imaging data to fine-tune monolingual and multilingual Transformer language models, enhancing their performance in brain encoding and multilingual downstream NLP tasks;
🎯 What it does: This paper proposes the FOND framework, deriving brain-like inference dynamics from variational inference and natural gradient, and applies it to construct an iterative variational autoencoder, particularly the iP-VAE recurrent neural network for sparse integer synaptic counts.
🎯 What it does: This study proposes a multi-brain-tuning method that enhances the alignment of a pre-trained speech model with human brain responses by jointly fine-tuning it on fMRI data from multiple subjects.
BraVE: Offline Reinforcement Learning for Discrete Combinatorial Action Spaces
Matthew Landers (University of Virginia), Afsaneh Doryab (University of Virginia)
CodeReinforcement Learning
🎯 What it does: We propose BraVE, a value estimation method for discrete combinatorial action spaces in offline reinforcement learning, which utilizes a tree structure for efficient search and value evaluation of the action space.
🎯 What it does: A batch mining method based on teacher model ranking and graph community detection (B3) is proposed, which enhances the training efficiency and performance of contrastive learning models by constructing batches that include strong negative samples.
Breaking the Compression Ceiling: Data-Free Pipeline for Ultra-Efficient Delta Compression
Xiaohui Wang (Fudan University), Tao Chen (Fudan University)
CodeCompressionTransformerLarge Language ModelImageTextMultimodality
🎯 What it does: A fully data-free extreme compression framework called UltraDelta is proposed, which achieves ultra-high compression rates for the delta weights of fine-tuned models while maintaining model performance.
🎯 What it does: A purely data-driven continuous spatiotemporal dynamics modeling framework called CoPS is proposed to predict the evolution of continuous physical fields from sparse, unstructured observations.
Breakthrough Sensor-Limited Single View: Towards Implicit Temporal Dynamics for Time Series Domain Adaptation
Mingyang Liu (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
CodeDomain AdaptationConvolutional Neural NetworkTime Series
🎯 What it does: The EDEN framework is proposed, which performs unsupervised domain adaptation on time series using multiple explicit domains (multi-scale, multi-subspace, multi-paragraph) and integrates three specialized modules to achieve more robust domain-invariant representations.
BridgePure: Limited Protection Leakage Can Break Black-Box Data Protection
Yihan Wang (University of Waterloo), Yaoliang Yu (University of Waterloo)
CodeClassificationRestorationSafty and PrivacyDiffusion modelImage
🎯 What it does: Proposes the BridgePure method, which utilizes a small amount of protective leakage for reverse mapping of black-box data protection to restore data availability.
🎯 What it does: This paper systematically evaluates the representation similarity (i.e., convergent learning) of different deep learning models under training processes, network architectures, and distribution shifts, and verifies its patterns through large-scale experiments.
Bridging Equivariant GNNs and Spherical CNNs for Structured Physical Domains
Colin Kohler (Northeastern University), Robin Walters (Northeastern University)
CodeSuper ResolutionConvolutional Neural NetworkGraph Neural NetworkReinforcement LearningContrastive LearningPoint CloudMeshPhysics Related
🎯 What it does: This paper proposes G2Sphere, a framework that utilizes SO(3) equivariant graph networks and spherical CNNs to perform encoding and decoding in Fourier space, directly mapping 3D geometry to continuous, high-frequency spherical signals, achieving zero-step super-resolution and efficient inference.
🎯 What it does: An Adaptive Unitary State Space Model (AUSSM) is proposed, achieving a balance between expressiveness and scalability for long sequences through input-dependent skew-symmetric recursion.
Bridging Human and LLM Judgments: Understanding and Narrowing the Gap
Felipe Maia Polo (University of Michigan), Yuekai Sun (University of Michigan)
CodeTransformerLarge Language ModelTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: A Bridge framework has been established to unify the modeling of human and LLM scoring, and through this model, achieve calibration of LLM scoring and quantitative analysis of systematic differences.
🎯 What it does: A two-dimensional spatially extended E/I network model was constructed, and various macroscopic dynamical states such as from no synchronization to synchronization, waves, spots, spikes, and chaos were quantitatively predicted through spectral theory (random matrix theory + Fourier transform);
🎯 What it does: Embedded rotation and scale equivariant convolutional layers into traditional CNNs, constructed and evaluated various symmetry-enhanced architectures to improve adversarial robustness without relying on adversarial training.
Bridging the gap to real-world language-grounded visual concept learning
Whie Jung (Korea Advanced Institute of Science and Technology), Seunghoon Hong (Korea Advanced Institute of Science and Technology)
CodeGenerationData SynthesisTransformerVision Language ModelDiffusion modelImage
🎯 What it does: A scalable visual concept learning framework is proposed, capable of adaptively discovering and aligning diverse language-specified concept axes in real-world scenarios, and achieving concept decoupling through combinatorial anchoring.
Bridging Time and Linguistics: LLMs as Time Series Analyzer through Symbolization and Segmentation
Jianyang Qin (Harbin Institute of Technology), Qing Liao (Harbin Institute of Technology)
CodeClassificationAnomaly DetectionRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTime SeriesSequential
🎯 What it does: By performing spectral symbolization and segment position encoding in the frequency domain, the time series is converted into a text format that can be input into LLMs, and pre-trained LLMs are utilized for tasks such as time series prediction and classification.
🎯 What it does: This paper studies the combination of the Segment Anything Model (SAM) and Digital Surface Model (DSM) for tree crown instance segmentation in high-resolution drone imagery, evaluating it in three ecological contexts: northern coniferous forests, temperate forests, and tropical rainforests.
Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
Brian Siyuan Zheng (University of Washington), Noah A. Smith (Allen Institute for AI)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study explores the robustness of language models when faced with non-canonical tokenization methods that were not seen during training, demonstrating that instruction-tuned models maintain a high level of performance; it also evaluates whether changing tokenization strategies during inference can enhance model performance.
CodeRobotic IntelligenceLarge Language ModelReinforcement LearningVision Language ModelMultimodality
🎯 What it does: Designed and implemented the Blink-Think-Link (BTL) framework and trained the BTL-UI GUI agent, significantly enhancing the perception, reasoning, and execution performance of GUI interactions.
🎯 What it does: A Buffer layer is proposed, which is a lightweight, pluggable auxiliary module for source-agnostic online testing adaptation, keeping the original backbone network unchanged;
🎯 What it does: This paper proposes a menu-based combinatorial auction design framework based on Continuous Regularized Flows (ODE) - BUNDLEFLOW, aimed at achieving revenue-optimal auctions that are DSIC in a single-buyer setting.
C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models
Amir Hossein Rahmati (Texas A&M University), Xiaoning Qian (Brookhaven National Laboratory)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes Contextual Low-Rank Adaptation (C-LoRA), which incorporates a data-dependent lightweight context module into the LoRA fine-tuning of LLMs, achieving scalable Bayesian uncertainty estimation.
🎯 What it does: This paper proposes C Prompt, a class-aware client knowledge interaction method for federated continual learning, which achieves knowledge consistency across clients through local class distribution compensation and class-aware prompt aggregation.
C3PO: Optimized Large Language Model Cascades with Probabilistic Cost Constraints for Reasoning
Antonios Valkanas (McGill University), Mark Coates (McGill University)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: A self-supervised LLM cascading reasoning framework C3PO is designed, which uses a confidence threshold to decide whether to exit early and controls reasoning costs through conformal prediction.
Yiyi Liu (Wuhan University of Technology), Biao Xiong (Wuhan University of Technology)
CodeTransformerPoint Cloud
🎯 What it does: A continuous perceptual edge (CAGE) network is proposed, which directly reconstructs vectorized indoor floor plans using 2D density maps projected from point clouds.
Calibrating Translation Decoding with Quality Estimation on LLMs
Di Wu (University of Amsterdam), Christof Monz (University of Amsterdam)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The paper achieves the calibration of the translation process of large language models by directly optimizing the likelihood of the translation model and the Pearson correlation with translation quality during the training phase, significantly improving translation quality and quality estimation capabilities.
CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs
Jan Hagnberger (University of Stuttgart), Mathias Niepert (University of Stuttgart)
CodeConvolutional Neural NetworkTransformerTime SeriesPhysics Related
🎯 What it does: This paper proposes a compressed latent space model CALM-PDE based on continuous adaptive convolution for efficiently solving arbitrary discrete time-varying partial differential equations.
Lingzhi Shen (University of Southampton), Shoaib Jameel (University of Southampton)
CodeTransformerLarge Language ModelMixture of ExpertsContrastive LearningTextMultimodality
🎯 What it does: Proposes the CALM framework, enabling large language models to self-perceive during inference and integrate cultural knowledge for cross-cultural adaptation.
CAMILA: Context-Aware Masking for Image Editing with Language Alignment
Hyunseung Kim (Samsung Semiconductor), Joon Hee Choi (Samsung Semiconductor)
CodeGenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageMultimodality
🎯 What it does: A context-aware image editing method called CAMILA is proposed, which utilizes a multimodal large language model to generate [MASK]/[NEG] tokens. It generates editing masks through a Token Broadcaster and Token Decoder, and achieves precise editing of executable instructions while ignoring non-executable ones using a diffusion model.
CamSAM2: Segment Anything Accurately in Camouflaged Videos
Yuli Zhou (ETH Zurich), Guolei Sun (Nankai University)
CodeSegmentationVideoMultimodality
🎯 What it does: This paper improves the performance of SAM2 in the task of video camouflage object segmentation by introducing learnable de-camouflage tokens, implicit and explicit object perception fusion modules, and an object prototype generation module, while keeping the original parameters of SAM2 unchanged.
CodeTransformerLarge Language ModelAgentic AITextBenchmark
🎯 What it does: This paper first conducts a manual analysis of 201 GitHub issues from 18 real LLM agent systems, constructing a classification system for agent system issues that includes 6 categories (compatibility, tools, memory, workflow, LLM operations, practicality) and 20 subcategories. Based on 50 reproducible issues, the AGENTISSUE-BENCH benchmark is manually built, and then automated repair experiments are conducted on this benchmark using three mainstream software engineering (SE) agents (SWE-agent, AutoCodeRover, Agentless) combined with GPT-4o and Claude-3.5-Sonnet.
CodeRetrievalGraph Neural NetworkLarge Language ModelTextGraphBenchmarkRetrieval-Augmented Generation
🎯 What it does: A new framework called GraphFlow is proposed, which achieves multi-step retrieval of text-rich knowledge graphs through the joint optimization of retrieval strategies and flow estimators.
Can Large Language Models Master Complex Card Games?
Wei Wang (Nankai University), Jie Tang (Tsinghua University)
CodeTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: By supervising the fine-tuning of LLM on high-quality game data, this study explores and verifies whether LLM can master eight complex card games, and examines its ability to master multiple games simultaneously as well as the impact on general capabilities.
CodeRepresentation LearningDrug DiscoveryTransformerLarge Language ModelPrompt EngineeringMultimodalityBiomedical Data
🎯 What it does: This paper proposes an unsupervised training framework for In-Context Representation Learning (ICRL), which can directly inject features from non-text modalities (such as molecular representations) into large language models (LLMs) during inference, enabling multimodal reasoning.
Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch?
Yijie Hu (Duke Kunshan University), Qiufeng Wang (Xi'an-Jiaotong Liverpool University)
CodeTransformerLarge Language ModelTextMultimodality
🎯 What it does: The research proposes a model merging method called IP-Merging, which transfers mathematical reasoning capabilities from specialized mathematical LLMs to multimodal LLMs.
Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework
Laura Kopf (Technische Universität Berlin), Oliver Eberle (Technische Universität Berlin)
CodeTransformerLarge Language ModelText
🎯 What it does: The PRISM framework is proposed to generate multi-concept descriptions for the internal features of large language models (LLMs), addressing the limitation of single descriptions that cannot capture polysemanticity.
Cascaded Language Models for Cost-Effective Human–AI Decision-Making
Claudio Fanconi (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
CodeTransformerLarge Language ModelReinforcement LearningTextBiomedical Data
🎯 What it does: An adaptive decision-making framework based on multi-level LLMs and human experts is proposed, achieving model delegation, fallback, and online learning through confidence and uncertainty thresholds, balancing accuracy, cost, and fallback.
🎯 What it does: A framework named CATransformers has been developed, capable of simultaneously optimizing the structure of Transformer models and the configuration of hardware accelerators during the early design phase to minimize total carbon emissions (including operational carbon and inherent carbon).
Causal Explanation-Guided Learning for Organ Allocation
Alessandro Marchese (Vrije Universiteit Brussel), Sam Verboven (Vrije Universiteit Brussel)
CodeOptimizationExplainability and InterpretabilityAdversarial AttackContrastive LearningTabularBiomedical Data
🎯 What it does: This paper presents CLEXNET, a causal explanation-guided model that utilizes directional information about the reasons for rejection during organ transplantation to improve acceptance predictions.
Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
Xiangning Yu (Tianjin University), Mengyue Yang (Hong Kong University of Science and Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: Proposes a chain-of-thought (CoT) optimization framework based on probabilistic necessary and sufficient (PNS) reasoning, which automatically eliminates redundant steps while maintaining or improving accuracy.
Causal-R: A Causal-Reasoning Geometry Problem Solver for Optimized Solution Exploration
Wenjun Wu (Xi'an Jiaotong University), Jun Liu (Lenovo Research)
CodeOptimizationExplainability and InterpretabilityComputational EfficiencyGraph Neural NetworkGraphBenchmark
🎯 What it does: The Causal-R model is proposed, which achieves efficient problem solving and multi-solution exploration for geometric problems through causal graph reasoning and forward matrix reasoning.
Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features
Kaichen Xu (Emory University), Xiaobo Sun (Emory University)
CodeRepresentation LearningTransformerContrastive LearningTabularBiomedical Data
🎯 What it does: Proposes the CAPE method, which provides location-aware encoding for Transformer to handle non-sequential but causally related features;
CCL: Causal-aware In-context Learning for Out-of-Distribution Generalization
Hoyoon Byun (Yonsei University), Kyungwoo Song (Yonsei University)
CodeDomain AdaptationRepresentation LearningTransformerLarge Language ModelAuto EncoderText
🎯 What it does: A causal-aware context learning (CCL) framework is proposed for out-of-distribution (OOD) scenarios, utilizing causal representations to select examples and enhance the generalization performance of large language models.
🎯 What it does: This study investigates the linear impact of initial noise perturbations in diffusion models on the generated results and proposes a Controlled and Constrained Sampling (CCS) method to achieve precise sampling under given target mean and mean squared error (MSE).
CellCLIP - Learning Perturbation Effects in Cell Painting via Text-Guided Contrastive Learning
MingYu Lu, Su-In Lee (University of Washington)
CodeRetrievalDrug DiscoveryTransformerContrastive LearningImageTextMultimodalityBiomedical Data
🎯 What it does: We propose CellCLIP, a cross-modal contrastive learning framework that uses natural language to encode perturbations and aligns perturbations with morphology by combining them with Cell Painting images, improving cross-modal retrieval and biological downstream tasks.
Centralized Reward Agent for Knowledge Sharing and Transfer in Multi-Task Reinforcement Learning
Haozhe Ma (National University of Singapore), Tze-Yun Leong (Nanyang Technological University)
CodeReinforcement Learning
🎯 What it does: The CenRA framework is proposed, which combines reward shaping with multi-task reinforcement learning. A centralized reward agent (CRA) distributes dense rewards to enhance learning efficiency in sparse reward environments and support new task transfer.
🎯 What it does: This study proposes a Vision Transformer model based on Masked Autoencoder, named ChA-MAEViT, for multi-channel images (MCI), aiming to enhance cross-channel feature learning and self-supervised representation.
🎯 What it does: A visual motion strategy based on reverse trajectory autoregression, called Chain-of-Action (CoA), is proposed, which achieves global-to-local reasoning by generating a complete action sequence in reverse from task keyframes.
Xiaohua Wang (Microsoft Research), Lili Qiu (Fudan University)
CodeTransformerLarge Language ModelText
🎯 What it does: The paper proposes a Chain-of-Model learning framework that embeds multi-scale information into the hidden layers of a Transformer, achieving a scalable multi-scale language model CoLM and CoLM-Air, which supports flexible inference, pre-fill acceleration, and continuous training.
Liang Wang (Microsoft Research), Furu Wei (Microsoft Research)
CodeGenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: A framework named CoRAG has been designed and implemented, allowing large language models to retrieve information step-by-step and dynamically rewrite queries to form a retrieval chain before generating answers.
Channel Matters: Estimating Channel Influence for Multivariate Time Series
Muyao Wang (Xidian University), James Kwok (Hong Kong University of Science and Technology)
CodeAnomaly DetectionGraph Neural NetworkTransformerTime Series
🎯 What it does: Proposed the Channel-wise Influence (ChInf) method, which quantifies the impact of each channel on model performance in multivariate time series, and based on this method, constructed two types of algorithms for anomaly detection and channel pruning.
ChartSketcher: Reasoning with Multimodal Feedback and Reflection for Chart Understanding
Muye Huang (Xi'an Jiaotong University), Jun Liu (Xi'an Jiaotong University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelReinforcement LearningVision Language ModelImageMultimodalityChain-of-Thought
🎯 What it does: This paper presents ChartSketcher, a chart understanding method that achieves multimodal step-by-step reasoning through programmatic drawing and visual feedback.
ChemOrch: Empowering LLMs with Chemical Intelligence via Groundbreaking Synthetic Instructions
Yue Huang (University of Notre Dame), Xiangliang Zhang (University of Notre Dame)
CodeDrug DiscoveryTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Designed and implemented the ChemOrch framework, which synthesizes instruction-response data in the field of chemistry through a two-stage process, and utilizes tools to ensure answer verifiability.
Thomas Norrenbrock (Leibniz Universität Hannover), Bodo Rosenhahn (Leibniz Universität Hannover)
CodeClassificationExplainability and InterpretabilityImage
🎯 What it does: A calibratable hierarchical explanation model CHiQPM is proposed, which can generate interpretable global class representations and hierarchical local explanations while maintaining extremely high prediction accuracy, and incorporates calibratable ensemble predictions.
Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Search
Haoran Sun (Shanghai Artificial Intelligence Laboratory), Xiaosong Wang (Shanghai Artificial Intelligence Laboratory)
CodeTransformerLarge Language ModelSupervised Fine-TuningMultimodalityBiomedical DataChain-of-Thought
🎯 What it does: This paper proposes the Mentor-Intern Collaborative Search (MICS) multi-model collaborative search framework, constructs a multi-modal medical reasoning dataset MMRP, and trains a multi-modal medical model Chiron-o1 with strong reasoning capabilities based on this dataset.
CodeClassificationRepresentation LearningBiomedical Data
🎯 What it does: ChromFound is proposed, a foundational model specifically designed for single-cell chromatin accessibility data, aimed at addressing the challenges of high-dimensional sparsity and dynamic chromatin landscapes.
CIDD: Collaborative Intelligence for Structure-Based Drug Design Empowered by LLMs
Bowen Gao (Tsinghua University), Yanyan Lan (Tsinghua University)
CodeDrug DiscoveryTransformerLarge Language ModelPrompt EngineeringBiomedical DataChain-of-Thought
🎯 What it does: A collaborative intelligent drug design framework (CIDD) has been designed and implemented, combining 3D molecular generation models with large language models (LLM). Through four modules of interactive analysis, design, reflection, and selection, it achieves a complete generation process from target binding pockets to drug candidate molecules.