International Conference on Learning Representations Β· 1682 papers
Broaden your SCOPE! Efficient Multi-turn Conversation Planning for LLMs with Semantic Space
Zhiliang Chen (National University of Singapore), Bryan Kian Hsiang Low (National University of Singapore)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: A framework called SCOPE is proposed for efficient dialogue planning in real-time multi-turn conversations using LLMs, without the need for additional LLM inference.
Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling
Minhyuk Seo (Seoul National University), Jonghyun Choi (Seoul National University)
CodeComputational EfficiencyLarge Language ModelImageMultimodality
π― What it does: A new online continual learning method is proposed under limited computation and storage budgets, balancing high performance and low resource consumption.
Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation
Abdelrahman Eldesokey (King Abdullah University of Science and Technology), Peter Wonka (King Abdullah University of Science and Technology)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: Proposes Build-A-Scene, an interactive 3D layout control method based on diffusion models, employing a multi-stage generation process, Dynamic Self-Attention (DSA), and a consistent 3D translation strategy;
π― What it does: Proposes the Bundle Neural Networks (BuNN) architecture, which learns graph neural networks by diffusing information over flat vector bundles;
C-CLIP: Multimodal Continual Learning for Vision-Language Model
Wenzhuo Liu (University of Chinese Academy of Sciences), Qi Tian (Huawei)
CodeRetrievalDomain AdaptationTransformerVision Language ModelContrastive LearningMultimodality
π― What it does: A multi-modal continuous learning framework named C-CLIP is proposed, which improves the catastrophic forgetting problem of visual-language models when continuously receiving new domain data.
Cached Multi-Lora Composition for Multi-Concept Image Generation
Xiandong Zou (Imperial College London), Yiren Zhao (Imperial College London)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: A training-free multi-LoRA combination framework called CMLoRA is proposed, which determines the injection order of LoRA by analyzing the high-frequency/low-frequency features of LoRA in the frequency domain, and employs a caching mechanism to enhance stability, addressing the semantic conflict issue in multi-concept image generation.
CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences
Ziran Qin (Shanghai Jiao Tong University), Jianguo Li (Ant Group)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: The CAKE framework is proposed, which combines adaptive KV cache allocation with hierarchical priority, hierarchical cascading management, and designs an eviction metric that considers dynamic time and space attention to efficiently manage KV caches in long text contexts.
Calibrating LLMs with Information-Theoretic Evidential Deep Learning
Yawei Li (LMU Munich), Mina Rezaei (LMU Munich)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes an Information Bottleneck-based Evidence Deep Learning (IB-EDL) method, which utilizes information bottleneck regularization to reduce the issues of overconfidence and poor calibration that arise when fine-tuning large language models on small datasets.
π― What it does: This paper proposes CAMEx, a sparse expert merging protocol that considers parameter curvature and implements a dynamic merging architecture.
π― What it does: This paper demonstrates through theoretical and experimental research that the current strongest unsupervised skill learning method, METRA, can be explained within the mutual information (MI) framework, and based on this, proposes a simpler contrastive learning + Successor Features (CSF) algorithm.
Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?
Egor Zverev (Institute of Science and Technology Austria), Christoph H. Lampert (Institute of Science and Technology Austria)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: This study investigates the separation of instructions and data in single-turn contexts for large language models (LLMs). It proposes a formal definition of separation, a computable empirical proxy metric, and releases the SEP dataset specifically for evaluating this metric. Subsequently, it assesses the separation and practicality of several mainstream LLMs and explores mitigation strategies such as prompt engineering, prompt optimization, and low-rank fine-tuning.
Xin Xu (Hong Kong University of Science and Technology), Yang Wang (Hong Kong University of Science and Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
π― What it does: This study investigates the ability of LLMs to solve mathematical word problems in longer contexts, constructs the E-GSM dataset, and proposes the CoLeG metric.
Zihao Zhou (University of California San Diego), Rose Yu (University of California San Diego)
CodeAnomaly DetectionTransformerLarge Language ModelPrompt EngineeringMultimodalityTime SeriesChain-of-Thought
π― What it does: The system evaluates the understanding and performance of LLM in time series anomaly detection, conducting experiments under various settings such as zero/few-shot, chain reasoning, and text/image input to validate six key hypotheses.
π― What it does: This paper proposes a method to facilitate the training of one modality model using the representation of another modality model (which may be incomplete or mismatched), achieving cross-modal collaborative learning in the absence of paired supervision.
Can Reinforcement Learning Solve Asymmetric Combinatorial-Continuous Zero-Sum Games?
Yuheng Li (William and Mary), Haipeng Chen (William and Mary)
CodeOptimizationReinforcement LearningGraph
π― What it does: This paper defines a new class of asymmetric combinatorial-continuous zero-sum games (ACCES) and proposes an algorithm based on the Combined Continuous Dual Oracle (CCDO) and its reinforcement learning implementation (CCDO-RL) to solve the Nash equilibria of these games.
Minghui Chen (University of British Columbia), Xiaoxiao Li (University of British Columbia)
CodeOptimizationFederated LearningSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes the FedTextGrad framework, which utilizes text gradients for iterative optimization of prompts in a federated learning environment.
Can Video LLMs Refuse to Answer? Alignment for Answerability in Video Large Language Models
Eunseop Yoon (Korea Advanced Institute of Science and Technology), Chang D. Yoo (Korea Advanced Institute of Science and Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningVideoText
π― What it does: A framework of 'answerability alignment' is proposed, enabling video large language models to judge and reject questions that exceed the scope of video information.
Can Watermarked LLMs be Identified by Users via Crafted Prompts?
Aiwei Liu (Tsinghua University), Xuming Hu (Hongkong University of Science and Technology)
CodeLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes a unified black-box detection method called Water-Probe, which can sample the same LLM multiple times through carefully designed prompts and compare the distribution differences of different watermark keys for similar prompts, thereby determining whether the LLM has embedded watermarks; it also introduces the Water-Bag scheme to enhance the undetectability of watermarks.
Can Watermarks be Used to Detect LLM IP Infringement For Free?
Zhengyue Zhao (University of Wisconsin-Madison), Chaowei Xiao (University of Wisconsin-Madison)
CodeTransformerLarge Language ModelText
π― What it does: This paper explores how to use watermarking technology of large language models (LLMs) to detect intellectual property infringement of models, proposing a new detection method called LIDet.
π― What it does: This paper demonstrates, through information-theoretic analysis and experimental research, the existence of 'label blindness' in unsupervised/self-supervised OOD detection methods without the use of labels, which leads to inevitable failure when ID and OOD share features. It proposes the Adjacent OOD evaluation task to validate this theory.
Can We Talk Models Into Seeing the World Differently?
Paul Gavrikov (Offenburg University), Janis Keuper (TΓΌbingen AI Center)
CodeTransformerPrompt EngineeringVision Language ModelImageTextMultimodality
π― What it does: This study investigates the performance of visual-language models (VLM) on texture/shape bias and explores the impact of natural language prompts on this bias.
Capability Localization: Capabilities Can be Localized rather than Individual Knowledge
Xiusheng Huang (Chinese Academy of Sciences), Kang Liu (Chinese Academy of Sciences)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This study evaluates the effectiveness of existing single knowledge localization methods through fidelity and reliability experimental systems and proves their ineffectiveness; subsequently, it proposes the 'Commonality Neuron Localization (CNL)' method, which locates shared neurons in similar datasets and verifies through fine-tuning, erasure, and cross-data experiments that these neurons are a collection of model capabilities, significantly enhancing or weakening the model's performance on different tasks.
CodePose EstimationGraph Neural NetworkTransformerLarge Language ModelImageTextGraph
π― What it does: A category-agnostic pose estimation method called CapeX is proposed, which utilizes a structured graph composed of text descriptions (text-graph) to locate key points in a query image without the need for supporting images.
CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation
Jie Liu (University of Amsterdam), Efstratios Gavves (University of Amsterdam)
CodeOptimizationRobotic IntelligenceTransformerLarge Language ModelTextBenchmark
π― What it does: Proposes the Cooperative Plan Optimization (CaPo) framework, which allows large language model-driven embodied multi-agent systems to generate long-term meta-plans through multi-round discussions before executing tasks, and dynamically adjust based on progress during execution.
CARTS: Advancing Neural Theorem Proving with Diversified Tactic Calibration and Bias-Resistant Tree Search
Xiao-Wen Yang (Nanjing University), Yu-Feng Li (Nanjing University)
CodeLarge Language ModelReinforcement LearningTextBenchmark
π― What it does: Designed and implemented the CARTS algorithm, which enhances the performance of first-order tree search in neural theorem proving through diversification strategies for calibration and bias-resistant value functions.
Catastrophic Failure of LLM Unlearning via Quantization
Zhiwei Zhang (Pennsylvania State University), Suhang Wang (Pennsylvania State University)
CodeLarge Language ModelText
π― What it does: The paper investigates the catastrophic failure of machine unlearning after quantization in LLMs and proposes a gradient significance-based unlearning framework with a large learning rate (SURE) to prevent quantization from recovering forgotten knowledge.
CATCH: Channel-Aware Multivariate Time Series Anomaly Detection via Frequency Patching
Xingjian Wu (East China Normal University), Bin Yang (Hong Kong University of Science and Technology)
CodeAnomaly DetectionTransformerTime Series
π― What it does: A multivariate time series anomaly detection framework named CATCH is proposed, which achieves joint detection of point anomalies and subsequence anomalies through frequency domain patching and a channel fusion module.
π― What it does: CatVTON is proposed, a virtual try-on system that achieves clothing and person image stitching in the spatial dimension using a simplified VAE+UNet structure.
Sueda Taner (ETH Zurich), Christoph Studer (ETH Zurich)
CodeOptimizationImage
π― What it does: A differentiable, automatically scaling, and non-false stationary regularization function constructed using the Cauchy-Schwarz inequality, referred to as the CS regularizer, is proposed. It can induce various structural properties (discrete value vectors, feature vectors, orthogonal column matrices, etc.) and is applied to tasks such as solving underdetermined linear equations and neural network weight quantization.
Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning
Gabriele Dominici (UniversitΓ della Svizzera italiana), Marc Langheinrich (UniversitΓ della Svizzera italiana)
CodeExplainability and InterpretabilityGraph Neural NetworkReinforcement LearningImage
π― What it does: A Causal Concept Graph Models (Causal CGMs) architecture is designed to address the causal ambiguity of deep learning models by learning interpretable causal graphs and high-dimensional concept representations, making the decision-making process traceable.
π― What it does: The DrBO method is proposed, which uses Bayesian optimization to find high-scoring DAGs, achieving sample-efficient causal graph learning.
Causal Graph Transformer for Treatment Effect Estimation Under Unknown Interference
Anpeng Wu (Zhejiang University), Kun Zhang (Zhejiang University)
CodeGraph Neural NetworkTransformerGraph
π― What it does: This paper proposes CauGramer, a causal graph transformer model for estimating treatment effects under unknown interference graphs; it aggregates L-order neighbor information through cross-attention and combines representation balancing with minimax moment conditions to achieve joint estimation of direct effects, peer effects, and total effects.
Causal Graphical Models for Vision-Language Compositional Understanding
Fiorenzo Parascandolo (University of Modena and Reggio Emilia), Rita Cucchiara (University of Modena and Reggio Emilia)
CodeGenerationRetrievalTransformerVision Language ModelImageTextMultimodality
π― What it does: This paper proposes a visual-language model (VLM) autoregressive training method called COGT based on causal graph models (CGM). It utilizes a dependency parser to pre-construct causal relationships between words, guiding semi-parallel word prediction, thereby enhancing the understanding and retrieval capabilities for language compositional tasks.
Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference
Aniket Vashishtha (University of Illinois Urbana-Champaign), Amit Sharma (Microsoft Research)
CodeTransformerLarge Language ModelPrompt EngineeringTabularBiomedical DataAlzheimer's Disease
π― What it does: Proposes using causal order instead of a complete causal graph as expert knowledge output, and designs a query method based on triplet prompts to improve the accuracy of obtaining causal structures from imperfect experts (LLMs and human annotators).
CausalRivers - Scaling up benchmarking of causal discovery for real-world time-series
Gideon Stein (Friedrich Schiller University Jena), Joachim Denzler (Friedrich Schiller University Jena)
CodeTime SeriesBenchmark
π― What it does: A real-world causal discovery benchmark based on river flow time series, CausalRivers, is proposed, which includes over 1000 monitoring stations, 15-minute resolution data, and provides subgraph sampling tools and three baselines.
Maxence Faldor (Imperial College London), Antoine Cully (Imperial College London)
CodeOptimizationComputational EfficiencyAuto EncoderTime Series
π― What it does: Developed and released the CAX library, achieving high-performance simulation and training of discrete, continuous, and neural cellular automata, demonstrating acceleration and new experiments in various tests.
π― What it does: This study proposes the CBGBench benchmark, unifying structure-based drug design (SBDD) and lead optimization tasks into a three-dimensional binding graph completion problem, and integrates 12 mainstream generative models with a unified evaluation framework;
π― What it does: A graph pre-training framework called CenPre is proposed, which enhances node representations through three modules: node-level and graph-level importance learning and representation alignment, thereby improving downstream tasks such as node classification, link prediction, and graph classification.
Isha Chaudhary (University of Illinois at Urbana-Champaign), Gagandeep Singh (University of Illinois at Urbana-Champaign)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Proposed and implemented the LLMCert-B framework, which can quantitatively certify counterfactual biases in large-scale LLM generated texts in a black-box manner and provide high-confidence unbiased probability intervals.
π― What it does: Designed and implemented a Conceptual Feedback Decoupling (CFD) method for learning robust universal molecular representations against distribution shifts.
π― What it does: This paper proposes CFG++, an improved Classifier-Free Guidance (CFG) method that utilizes uncertain noise for interpolation rather than extrapolation during the denoising step, resulting in smoother and higher-quality generation and reverse sampling results in text-guided diffusion models.
π― What it does: Proposes the Chain-of-Focus (CoF) prompting method for achieving visual step-by-step reasoning and adaptive prompting in large autoregressive visual models (LAVMs);
Chain-of-Thought Provably Enables Learning the (Otherwise) Unlearnable
Chenxiao Yang (Toyota Technological Institute at Chicago), David Wipf (Amazon Web Services)
CodeTransformerLarge Language ModelTextChain-of-Thought
π― What it does: This study investigates the theory and practice of Chain-of-Thought (CoT) in enhancing learning effectiveness through task decomposition in the context of language models.
ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understanding
Zhengzhuo Xu (International Digital Economy Academy), Jian Guo (Peking University)
CodeTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsVision Language ModelMultimodalityTabular
π― What it does: This paper proposes ChartMoE, a multimodal large language model that uses Mixture of Experts (MoE) as a visual-language connector to enhance chart understanding and reasoning capabilities.
CheapNet: Cross-attention on Hierarchical representations for Efficient protein-ligand binding Affinity Prediction
Hyukjun Lim (Seoul National University), Sangseon Lee (Inha University)
CodeDrug DiscoveryGraph Neural NetworkBiomedical Data
π― What it does: We propose CheapNet, a protein-ligand binding affinity prediction model that integrates atomic-level embeddings with hierarchical representations after differentiable clustering aggregation through cross-attention fusion.
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
Xiaosen Zheng (Sea AI Lab), Min Lin (Sea AI Lab)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: The study demonstrates how to use the 'empty model' to deceive automated LLM evaluation benchmarks through structured fake responses, resulting in a high win rate.
ChemAgent: Self-updating Memories in Large Language Models Improves Chemical Reasoning
Xiangru Tang (Yale University), Mark Gerstein (Yale University)
CodeDrug DiscoveryTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: A self-updating chemical reasoning library (ChemAgent) has been constructed, which breaks down complex problems into subtasks and stores the subtasks and their solutions in multiple types of memory (planning, execution, knowledge), achieving continuous learning and improvement of the LLM.
π― What it does: Using a non-differentiable quantum chemistry oracle (such as GFN2-xTB) to guide a 3D molecular diffusion model through zero-order gradient estimation, achieving conditional molecular generation and enhancing molecular stability.
CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs
Jinlan Fu (National University of Singapore), See-Kiong Ng (National University of Singapore)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextMultimodality
π― What it does: Optimizes the hallucination problem of multimodal large language models and proposes a cross-modal hierarchical direct preference optimization method called CHiP.
ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains
Yein Park (Korea University), Jaewoo Kang (Korea University)
CodeTransformerLarge Language ModelPrompt EngineeringTextTime SeriesBiomedical DataBenchmark
π― What it does: Proposes the CHROKNOWBENCH and CHROKNOWLEDGE frameworks to evaluate the knowledge retention of large language models across different domains over time, and enhances the accuracy of memory for invariant objects through time series reasoning using CHROKNOWPROMPT.
Yanhong Li (University of Chicago), Jiawei Zhou (Stony Brook University)
CodeGenerationDomain AdaptationKnowledge DistillationTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: The Chunk-Distilled Language Modeling (CD-LM) method is proposed, which efficiently generates multiple tokens and injects new knowledge without additional training by retrieving multi-word chunks during the generation process and making chunk-level acceptance/rejection decisions.
CipherPrune: Efficient and Scalable Private Transformer Inference
Yancheng Zhang (University of Central Florida), Qian Lou (University of Central Florida)
CodeSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The CipherPrune framework is proposed, achieving adaptive layer-wise token pruning and multi-order polynomial approximation for encrypted inputs, significantly improving the inference efficiency of private Transformers.
Circuit Representation Learning with Masked Gate Modeling and Verilog-AIG Alignment
Haoyuan WU, Bei Yu (Chinese University of Hong Kong)
CodeRepresentation LearningGraph Neural NetworkLarge Language ModelTextGraph
π― What it does: This paper proposes a constrained mask modeling paradigm MGVGA for circuit representation learning, which can learn fine-grained structural information and abstract functional information without compromising logical equivalence.
Circuit Transformer: A Transformer That Preserves Logical Equivalence
Xihan Li (University College London), Jun Wang (Huawei)
CodeGenerationOptimizationTransformerGraph
π― What it does: This study investigates how to ensure the logical equivalence of logic circuit implementations under generative neural networks, proposing the Circuit Transformer to generate strictly equivalent and more compact circuits.
CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer
Yang Liu (Hong Kong University of Science and Technology), Jia Li (Hong Kong University of Science and Technology)
CodeTransformerTime Series
π― What it does: A direct prediction S2S climate forecasting model named CirT has been designed and trained, capable of outputting multivariate averages over intervals of 2β6 weeks.
CityAnchor: City-scale 3D Visual Grounding with Multi-modality LLMs
Jinpeng Li (Wuhan University), Bisheng Yang (Wuhan University)
CodeRecognitionObject DetectionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityPoint Cloud
π― What it does: This paper proposes CityAnchor, a two-stage urban-scale 3D visual localization method based on a multimodal large language model. It first performs coarse localization of candidate areas on a 2D projection map, and then conducts fine-grained matching of text and 3D point clouds within the candidate areas to ultimately locate the target object.
CL-DiffPhyCon: Closed-loop Diffusion Control of Complex Physical Systems
Long Wei (Westlake University), Tailin Wu (Westlake University)
CodeReinforcement LearningDiffusion modelTime SeriesPhysics Related
π― What it does: To address the closed-loop control problem of complex physical systems, the CL-DiffPhyCon method is proposed, which generates control sequences under real-time feedback through an asynchronous diffusion model, significantly reducing sampling costs.
CL-MFAP: A Contrastive Learning-Based Multimodal Foundation Model for Molecular Property Prediction and Antibiotic Screening
Gen Zhou (Western University), Pingzhao Hu (Western University)
CodeRepresentation LearningDrug DiscoveryGraph Neural NetworkTransformerContrastive LearningMultimodalityGraphBiomedical Data
π― What it does: This paper proposes the CL-MFAP model, which utilizes contrastive learning pre-training with three modalities: SMILES, molecular graphs, and Morgan fingerprints, followed by fine-tuning on downstream tasks such as antibacterial activity.
ClawMachine: Learning to Fetch Visual Tokens for Referential Comprehension
Tianren Ma (University of Chinese Academy of Sciences), Qixiang Ye (University of Chinese Academy of Sciences)
CodeRecognitionObject DetectionSegmentationTransformerLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: ClawMachine is proposed, a multimodal large language model that directly uses a set of visual tokens for reference understanding and localization without requiring additional syntax.
π― What it does: The CLDyB framework is proposed, which dynamically generates task sequences for pre-trained model-based continual learning methods using Markov decision processes and Monte Carlo tree search, providing a unified and repeatable challenging benchmark for evaluating various CL methods.
CLIPDrag: Combining Text-based and Drag-based Instructions for Image Editing
Ziqi Jiang, Long Chen
CodeDiffusion modelImage
π― What it does: By combining text descriptions with user-dragged points, the CLIPDrag scheme is proposed to achieve precise and unambiguous image editing.
π― What it does: A zero-shot classification method called CLIPure for adversarial purification in the CLIP multimodal latent space has been developed.
π― What it does: The CLoSD system is proposed, which implements text-based multi-task role control, combining a real-time diffusion planner (DiP) with a physical simulation reinforcement learning tracking controller to perform interactive actions such as navigation, striking, sitting, and standing in a physical environment.
π― What it does: This paper proposes CO-MOT, which utilizes a combination of CO-opetition Label Assignment and Shadow Set to enhance the synergy between detection and tracking in a Transformer-based end-to-end MOT model, significantly improving issues related to tracking termination and loss.
COAT: Compressing Optimizer states and Activations for Memory-Efficient FP8 Training
Haocheng Xi (University of California), Song Han
CodeCompressionOptimizationComputational EfficiencyTransformerLarge Language ModelVision Language ModelTextMultimodality
π― What it does: COAT is proposed, a training framework that quantizes optimizer states and activations to FP8, significantly reducing memory usage and improving speed for large model training.
CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs
Dung Manh Nguyen, Nghi D. Q. Bui (FPT Software AI Center)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: A large-scale multiple-choice question benchmark called CodeMMLU is proposed to evaluate the capabilities of CodeLLM in code understanding and reasoning.
π― What it does: COFlowNet is proposed, a generative flow network for offline environments that can generate diverse and high-scoring candidate substances without relying on online evaluation.
CogCoM: A Visual Language Model with Chain-of-Manipulations Reasoning
Ji Qi (Tsinghua University), Jie Tang (Tsinghua University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: Proposed and implemented the Chain of Manipulations (CoM) mechanism, enabling visual language models to gradually reason and provide answers through a series of interpretable visual operations such as actively annotating, scaling, counting, and calculating images.
π― What it does: CogVideoX is a text-to-video generation model based on a diffusion Transformer, capable of generating high-quality videos of up to 10 seconds in length, with coherent actions and semantic consistency at 16fps and a resolution of 768Γ1360.
CoInD: Enabling Logical Compositions in Diffusion Models
Sachit Gaudi (Michigan State University), Vishnu Boddeti
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: This study investigates the generation of arbitrary logical attribute combinations in diffusion models and proposes the COIND objective, which enforces conditional independence to enhance the logical consistency and diversity of samples.
CollabEdit: Towards Non-destructive Collaborative Knowledge Editing
Jiamu Zheng (Zhejiang University), Tao Lin (Westlake University)
CodeTransformerLarge Language ModelText
π― What it does: The COLLABEDIT framework is proposed, enabling multiple parties to collaboratively edit knowledge in large language models without disclosing editing requests.
Collaborative Discrete-Continuous Black-Box Prompt Learning for Language Models
Hualin Zhang (Mohamed bin Zayed University of Artificial Intelligence), Yi Chang (Jilin University)
CodeOptimizationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: By jointly optimizing discrete and continuous prompts, a ZO-PoG black-box prompt learning framework is proposed, which can enhance the performance of large language models on downstream tasks without accessing the internal parameters of the model.
π― What it does: This paper studies the phenomenon of 'Neural Collapse' in neural networks within language models, finding that debiased models tend to collapse more in the word vectors and token representations of gender-related words. Based on this, it proposes the addition of the (Uβ―NC)β―3 regularization term during the fine-tuning process, forming a general, low-cost debiasing method; it also makes plug-in improvements to various debiasing strategies (MABEL, ASE, BEC, etc.).
Combatting Dimensional Collapse in LLM Pre-Training Data via Submodular File Selection
Ziqing Fan (Shanghai Jiao Tong University), Yanfeng Wang (Shanghai Jiao Tong University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A document selection algorithm based on feature decorrelation, DiSF, is proposed to avoid the dimensional collapse problem caused by domain relevance during LLM pre-training.
π― What it does: A conservative entropy minimization (COME) method based on subjective logic is proposed to address the model collapse issue caused by overconfidence in traditional entropy minimization during test-time adaptation (TTA).
ComLoRA: A Competitive Learning Approach for Enhancing LoRA
Qiushi Huang (Southern University of Science and Technology), Yu Zhang (Southern University of Science and Technology)
CodeLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
π― What it does: Combine competitive learning to train multiple LoRA modules, and only use the optimal LoRA during the inference phase to enhance the expressiveness and performance of LoRA.
Wenting Zhao (Cornell University), Alexander M Rush (Cornell University)
CodeAI Code AssistantLarge Language ModelTextBenchmark
π― What it does: Proposes the COMMIT0 benchmark, which requires AI to implement a complete Python library from scratch using natural language specifications and unit tests.
Comparing noisy neural population dynamics using optimal transport distances
Amin Nejatbakhsh (Flatiron Institute), David Lipshutz (Baylor College of Medicine)
CodeDiffusion modelTime Series
π― What it does: This paper proposes a new metric method - Causal Optimal Transport (Causal OT) distance, for comparing neural system trajectories with noise and dynamic characteristics.
Competing Large Language Models in Multi-Agent Gaming Environments
Jen-tse Huang (Chinese University of Hong Kong), Michael Lyu
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: This paper presents Ξ³-Bench, a multi-player, multi-round, multi-action game evaluation framework designed to quantify the performance of large language models in complex decision-making scenarios.
Complementary Label Learning with Positive Label Guessing and Negative Label Enhancement
Yuhang Li (Southeast University), Yuheng Jia (Southeast University)
CodeClassificationImage
π― What it does: A novel complementary label learning framework PLNL is proposed, which infers complete labels through two steps: Positive Label Guessing (PLG) and Negative Label Enhancement (NLE), training multi-classifiers from weakly supervised complementary labels.
Composing Unbalanced Flows for Flexible Docking and Relaxation
Gabriele Corso (Massachusetts Institute of Technology), Andreas Krause (ETH Zurich)
CodeProtein Structure PredictionFlow-based ModelBiomedical Data
π― What it does: A general flow matching framework called Unbalanced Flow Matching (UFM) is proposed to address the issues of protein flexibility and non-physical pose generation in molecular docking, and a flexible docking method FLEXDOCK is constructed based on this framework.
Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering
Xingrui Wang (Johns Hopkins University), Alan Yuille (Johns Hopkins University)
CodeObject DetectionObject TrackingData SynthesisConvolutional Neural NetworkRecurrent Neural NetworkReinforcement LearningVision Language ModelVideoPhysics Related
π― What it does: Proposed the DynSuperCLEVR dataset and the NS-4DPhysics model for understanding and reasoning about the four-dimensional dynamic properties of 3D objects, such as speed, acceleration, and collisions, in video question answering.
Compositional Entailment Learning for Hyperbolic Vision-Language Models
Avik Pal (University of Amsterdam), Pascal Mettes (University of Amsterdam)
CodeObject DetectionRetrievalRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: A new framework for visual-text representation learning in hyperbolic space, HyCoCLIP, is proposed by introducing the hierarchical relationship between image local boxes and corresponding text segments, using hierarchical contrastive loss and hierarchical entailment loss for joint training.
CodeOptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper studies data selection in the context of fine-tuning LLMs under computational constraints, proposing a computation-aware data selection objective and cost model. Systematic experiments and performance modeling are conducted across various model sizes, tasks, and data budgets, indicating that traditional high-cost methods are not computationally optimal under most budgets.
CoMRes: Semi-Supervised Time Series Forecasting Utilizing Consensus Promotion of Multi-Resolution
Yunju Cho (Seoul National University), Jay-Yoon Lee (Seoul National University)
CodeOptimizationTransformerTime Series
π― What it does: Proposes the CoMRes model, which utilizes multi-scale self-supervised learning and consistency to enhance long-term time series forecasting.
Chung-En Sun (University of California San Diego), Tsui-Wei Weng (University of California San Diego)
CodeClassificationGenerationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes Concept Bottleneck Large Language Models (CB-LLMs), which achieve self-explanatory text classification and generation models by incorporating an interpretable concept bottleneck layer into pre-trained LLMs.
π― What it does: A concept elimination method (CPE) is proposed in text-to-image diffusion models, which can accurately remove target concepts while retaining diverse remaining concepts.
CONDA: Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts
Jihye Choi (University of Wisconsin Madison), Somesh Jha (University of Wisconsin Madison)
CodeClassificationDomain AdaptationExplainability and InterpretabilityTransformerContrastive LearningImageBiomedical Data
π― What it does: This paper presents CONDA, an adaptive framework based on Concept Bottleneck (CBM) designed to enhance the robustness and accuracy of interpretable classifiers based on large foundation models (FM) in distribution shift environments during deployment (testing).
Conformal Language Model Reasoning with Coherent Factuality
Maxon Rubin-Toles (University of Pennsylvania), Surbhi Goel (University of Pennsylvania)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: For reasoning tasks, a method is proposed to ensure 'Coherent Factuality' in the argument steps generated by language models, along with a filtering algorithm based on segmented conformal prediction.
Botong Zhang (University of Pennsylvania), Osbert Bastani (University of Pennsylvania)
CodeClassificationOptimizationImageSequential
π― What it does: A general conformal structured prediction framework is proposed, capable of constructing prediction sets with coverage guarantees in any structured label space (such as DAGs, trees, etc.);
Jeremy Carleton (Texas A and M University), Aditya Akella (University of Texas at Austin)
CodeOptimizationReinforcement LearningTabular
π― What it does: This paper proposes a sublinear scheduling algorithm (CONGO) that utilizes compressed sensing techniques for gradient estimation in zero-order online convex optimization (OCO) problems with sparse gradients, achieving sample efficiency and dimension-independent performance.
Daniel Paleka (ETH Zurich), Florian Tramèr (ETH Zurich)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: This paper proposes a method to instantaneously evaluate language model predictors (LLM forecasters) using consistency checks (such as negation, synonyms, consequences, etc.), constructs an automated evaluation pipeline, and generates a long-term consistency benchmark.
Constraint-Conditioned Actor-Critic for Offline Safe Reinforcement Learning
Zijian Guo (Boston University), Wenchao Li (Boston University)
CodeSafty and PrivacyReinforcement LearningTabular
π― What it does: The Constraint-Conditioned Actor-Critic (CCAC) method is proposed to learn adaptive and robust policies under different cost thresholds in offline safe reinforcement learning, modeling the state-action distribution through constraint conditions to address the safety-performance imbalance caused by out-of-distribution (OOD) state-action.
Constructing Confidence Intervals for Average Treatment Effects from Multiple Datasets
Yuxin Wang (Ludwig Maximilian University of Munich), Stefan Feuerriegel (Ludwig Maximilian University of Munich)
CodeBiomedical DataElectronic Health Records
π― What it does: A method based on Predictive Enhanced Inference (PPI) is proposed to construct confidence intervals for the Average Treatment Effect (ATE) from two observational datasets (one small and unbiased, the other large and potentially subject to unobserved confounding).
Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data
Michael Wornow (Stanford University), Nigam Shah
CodeTransformerLarge Language ModelTabularBiomedical DataElectronic Health RecordsBenchmark
π― What it does: This study investigates the impact of different context lengths on clinical prediction tasks using electronic health records (EHR) and systematically evaluates the effects of three EHR-specific attributes (copy-forwarding, irregular time intervals, disease progression) on model performance.