ICML 2024 Papers — Page 4
International Conference on Machine Learning · 2610 papers
Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delays
Qingyuan Wu (University of Liverpool), Chao Huang (University of Southampton)
Reinforcement LearningSequential
🎯 What it does: This paper proposes an Auxiliary-Delayed Reinforcement Learning (AD-RL) framework that utilizes short-delay auxiliary tasks to accelerate the learning of the original long-delay tasks, while ensuring performance and reducing sample complexity.
Bootstrap AutoEncoders With Contrastive Paradigm for Self-supervised Gaze Estimation
Yaoming Wang (Shanghai Jiao Tong University), Hongkai Xiong (Shanghai Jiao Tong University)
Pose EstimationRepresentation LearningConvolutional Neural NetworkTransformerAuto EncoderContrastive LearningImage
🎯 What it does: A self-supervised full-face gaze estimation framework BeCa is proposed, which combines autoencoders with contrastive learning, and designs the InfoMSE loss for end-to-end unsupervised training.
Bootstrapping Fisher Market Equilibrium and First-Price Pacing Equilibrium
Luofeng Liao (Columbia University), Christian Kroer (Columbia University)
OptimizationTabularFinance Related
🎯 What it does: This paper proposes a Bootstrap inference method for two economic models: Linear Fisher Market Equilibrium (LFM) and First-Price Auction Equilibrium (FPPE), proving its consistency and providing a specific implementation scheme.
Borda Regret Minimization for Generalized Linear Dueling Bandits
Yue Wu (University of California), Quanquan Gu (University of California)
OptimizationTime Series
🎯 What it does: This paper studies the Borda reward minimization problem under the generalized linear dueling bandits framework, proposing two algorithms: Explore-Then-Committ (BETC-GLM) and EXP3-type (BEXP3), and provides theoretical upper and lower bounds.
BOtied: Multi-objective Bayesian optimization with tied multivariate ranks
Ji Won Park (Genentech), Kyunghyun Cho (New York University)
OptimizationDrug DiscoveryTabularBiomedical DataBenchmark
🎯 What it does: Proposes a BOtied multi-objective Bayesian optimization algorithm based on multivariate cumulative distribution functions.
Bottleneck-Minimal Indexing for Generative Document Retrieval
Xin Du (Waseda University), Kumiko Tanaka-Ishii (Waseda University)
GenerationRetrievalTransformerLarge Language ModelText
🎯 What it does: This paper reconsiders Generative Document Retrieval (GDR) as an information bottleneck problem and proposes a query distribution-based indexing design method called Bottleneck-Minimal Indexing (BMI).
Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints
Yunsheng Tian (Massachusetts Institute of Technology), Mina Konakovic Lukovic (Massachusetts Institute of Technology)
Optimization
🎯 What it does: A Bayesian optimization method under unknown physical constraints, BE-CBO, is proposed, specifically designed to handle infeasible samples where the objective function values cannot be obtained, focusing on the exploration and exploitation of constraint boundaries.
Bounded and Uniform Energy-based Out-of-distribution Detection for Graphs
Shenzhi Yang (Soochow University), Xiaofang Zhang (Soochow University)
Anomaly DetectionGraph Neural NetworkGraph
🎯 What it does: The NODESAFE method is proposed, which introduces two constraints (L_bound and L_uniform) during training to bound and uniform the negative energy scores of GNNs, thereby enhancing the OOD detection capability at the node level.
Bounding the Excess Risk for Linear Models Trained on Marginal-Preserving, Differentially-Private, Synthetic Data
Yvonne Zhou (University of Maryland), Dana Dachman-Soled (University of Maryland)
Data SynthesisSafty and PrivacyTabular
🎯 What it does: The study investigates the excess risk of training linear models on differentially private synthetic data while maintaining low-order marginal distributions, providing theoretical upper and lower bounds.
Box Facets and Cut Facets of Lifted Multicut Polytopes
Lucas Fabian Naumann, Bjoern Andres (Technische Universität Dresden)
OptimizationGraph
🎯 What it does: This study investigates the geometric properties of the lifted multicut polytope, proving when the lower box inequalities form the facets of the polytope and demonstrating that the facetization problem for cutting inequalities is NP-hard.
Boximator: Generating Rich and Controllable Motions for Video Synthesis
Jiawei Wang (ByteDance Research), Hang Li (ByteDance Research)
GenerationData SynthesisDiffusion modelVideo
🎯 What it does: Introducing Boximator, a pluggable bounding box constraint module that enables fine-grained motion control of multiple objects within existing video diffusion models.
BRAIn: Bayesian Reward-conditioned Amortized Inference for natural language generation from feedback
Gaurav Pandey (IBM Research), Ramón Fernandez Astudillo (IBM Research)
GenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A new RLHF algorithm based on distribution matching, called BRAIN, is proposed. It constructs the target distribution using a Bayesian approach and employs a self-normalizing baseline to reduce gradient variance, achieving efficient alignment for large language models.
Breadth-First Exploration on Adaptive Grid for Reinforcement Learning
Youngsik Yoon (POSTECH), Jungseul Ok (POSTECH)
Graph Neural NetworkReinforcement LearningSequential
🎯 What it does: This paper proposes a breadth-first exploration method based on adaptive grids (BEAG) for systematically managing achieved and unachieved sub-goals in sparse long-horizon goal-conditioned reinforcement learning, significantly improving sample efficiency.
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Yichao Fu (University of California San Diego), Hao Zhang (University of California San Diego)
OptimizationComputational EfficiencyAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: Proposes LOOKAHEAD DECODING, a method for accelerating LLM inference by generating multi-word n-grams using parallel Jacobi iteration without the need for auxiliary draft models, and validating in the same step.
Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agents
Chung-En Sun (University of California San Diego), Tsui-Wei Weng (University of California San Diego)
Reinforcement LearningVideo
🎯 What it does: A stochastic smooth robust training algorithm for reinforcement learning, S-DQN and S-PPO, is proposed to enhance clean rewards and robust performance.
Breaking through the learning plateaus of in-context learning in Transformer
Jingwen Fu (Xi'an Jiaotong University), Nanning Zheng (Xi'an Jiaotong University)
TransformerAuto EncoderText
🎯 What it does: This study investigates the phenomenon of learning plateau that occurs during the in-context learning process of Transformers and proposes a method to overcome the plateau by improving the weights component of the model.
Bridging Data Gaps in Diffusion Models with Adversarial Noise-Based Transfer Learning
Xiyu Wang (University of Sydney), Chang Xu (University of Sydney)
GenerationDomain AdaptationDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: A few-shot transfer learning method based on diffusion models, DPMs-ANT, is proposed, which significantly improves the quality and diversity of few-shot image generation by combining similarity-guided training and adversarial noise selection.
Bridging discrete and continuous state spaces: Exploring the Ehrenfest process in time-continuous diffusion models
Ludwig Winkler (Technical University of Berlin), Manfred Opper (University of Birmingham)
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelScore-based ModelImageStochastic Differential Equation
🎯 What it does: This study investigates the time-reversal relationship between discrete state Markov jump processes and continuous state SDEs, proposes a loss function based on conditional expectation, and verifies its feasibility in image generation tasks through the correspondence between the Ehrenfest process and the Ornstein-Uhlenbeck process.
Bridging Environments and Language with Rendering Functions and Vision-Language Models
Theo Cachet, Olivier Sigaud (Institute of Intelligent Systems and Robotics)
Robotic IntelligenceTransformerReinforcement LearningVision Language ModelImageText
🎯 What it does: This paper proposes to decompose the Language Conditioned Agent (LCA) into two steps: ① using a Vision-Language Model (VLM) and a rendering function to retrieve environment configurations with high similarity to the given text in the configuration space; ② employing a pre-trained Goal Conditioned Reinforcement Learning (GCRL) policy to achieve the retrieved configurations, thereby enabling zero-shot language-guided task execution.
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses
Panagiotis Koromilas (National and Kapodistrian University of Athens), Yannis Panagakis (National and Kapodistrian University of Athens)
OptimizationRepresentation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper provides a theoretical analysis of different contrastive learning losses (InfoNCE family), proving that they share the same optimal solution in finite batches and asymptotic cases, and proposes a contrastive energy loss that separates positive and negative samples (DHEL), as well as the optimality of kernel-based contrastive learning (KCL) in non-asymptotic cases.
Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning
Jiaqi Wang (Pennsylvania State University), Fenglong Ma (Pennsylvania State University)
Federated LearningKnowledge DistillationImage
🎯 What it does: Designed the FedType framework, which achieves heterogeneous model aggregation through proxy models without the need for common data;
Bring Your Own (Non-Robust) Algorithm to Solve Robust MDPs by Estimating The Worst Kernel
Uri Gadot (Technion), Shie Mannor (NVIDIA Research)
Reinforcement Learning
🎯 What it does: The EWoK method is proposed, which approximates the worst transition kernel during training by resampling the nominal transition kernel, allowing any non-robust RL algorithm to learn robust policies.
Bringing Motion Taxonomies to Continuous Domains via GPLVM on Hyperbolic manifolds
Noémie Jaquier (Karlsruhe Institute of Technology), Tamim Asfour (Karlsruhe Institute of Technology)
ClassificationRepresentation LearningVideo
🎯 What it does: This paper maps human action taxonomy to continuous space, using a Gaussian Process Hidden Variable Model on hyperbolic geometry (GPHLVM) to learn embeddings.
Building Socially-Equitable Public Models
Yejia Liu (University of California), Shaolei Ren (University of California)
Recurrent Neural NetworkTransformerTabularTime Series
🎯 What it does: This paper proposes a fair objective for public models (Equitable Objective), enabling the model to achieve a more balanced distribution of decision costs when providing predictions for diverse downstream agents, and presents training algorithms applicable to both differentiable and non-differentiable costs.
BWS: Best Window Selection Based on Sample Scores for Data Pruning across Broad Ranges
Hoyong Choi (Samsung Research), Hye Won Chung (Korea Advanced Institute of Science and Technology)
ClassificationData-Centric LearningConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A general and efficient data subset selection method called Best Window Selection (BWS) is proposed, which achieves data pruning by sorting based on difficulty scores and selecting the best subset within a window.
By Tying Embeddings You Are Assuming the Distributional Hypothesis
Francesco Bertolotti (Università degli Studi di Milano), Walter Cazzola (Università degli Studi di Milano)
TransformerText
🎯 What it does: This study investigates the theoretical and experimental effects of input-output embedding weight sharing on language models, clarifying its relationship with the distribution hypothesis.
ByMI: Byzantine Machine Identification with False Discovery Rate Control
Chengde Qian (Nankai University), Changliang Zou (Nankai University)
Anomaly DetectionFederated LearningImage
🎯 What it does: A ByMI method is proposed, which detects Byzantine machines through sample splitting, robust mean estimation, and symmetric score statistics, while achieving FDR control in the process.
Byzantine Resilient and Fast Federated Few-Shot Learning
Ankit Pratap Singh (Iowa State University), Namrata Vaswani (Iowa State University)
OptimizationFederated LearningTabular
🎯 What it does: In a federated learning environment, a fast and sample-efficient few-shot learning algorithm resistant to Byzantine attacks, Byz-AltGDmin, is proposed, based on AltGDmin and robust aggregation using geometric medians.
Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updates
Youssef Allouah (École Polytechnique Fédérale de Lausanne), Sasha Voitovych (University of Toronto)
Federated LearningConvolutional Neural NetworkImage
🎯 What it does: This paper introduces robust aggregation in federated learning, analyzing and proving that client subsampling and local updates affect convergence in the presence of Byzantine clients. It provides sufficient conditions for the subsampling size and Byzantine upper limit that ensure convergence. Furthermore, it proves that when the sampling size is sufficiently large, the system achieves near-optimal convergence error, which decreases with the number of local updates. The theory is validated through experiments, demonstrating the robustness of FedRo under different attacks.
C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models
Mintong Kang (University of Illinois at Urbana-Champaign), Bo Li (University of Chicago)
GenerationRetrievalTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes the C-RAG framework, aimed at providing certified generation risk for retrieval-augmented generation (RAG) models. It investigates whether RAG can reduce generation risk, how to provide provable risk guarantees, and the sufficient conditions under which RAG models can lower generation risk.
Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling
Yair Schiff (Cornell University), Volodymyr Kuleshov (Cornell University)
TransformerSupervised Fine-TuningBiomedical Data
🎯 What it does: The Caduceus DNA foundational model is proposed, utilizing BiMamba and MambaDNA to achieve bidirectional and reverse complement (RC) equivariant long-range DNA sequence modeling, demonstrating excellent performance in genomic tasks such as variant effect prediction.
Calibration Bottleneck: Over-compressed Representations are Less Calibratable
Deng-Bao Wang (Southeast University), Min-Ling Zhang (Southeast University)
ClassificationRepresentation LearningConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This study investigates the calibratability of deep neural networks, analyzing the negative impact of weight decay on model calibration. By performing linear probing and information bottleneck analysis on intermediate layer features, it proposes a Progressive Layer Pruning (PLP) method to enhance the calibration performance of the model.
CaM: Cache Merging for Memory-efficient LLMs Inference
Yuxin Zhang (Xiamen University), Rongrong Ji (Xiamen University)
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A cache merging (CaM) method is proposed, which utilizes attention weights to merge evicted KV caches into retained caches, reducing memory usage and output disturbance.
Can a Few Decide for Many? The Metric Distortion of Sortition
Ioannis Caragiannis (Aarhus University), Jannik Peters (TU Berlin)
Tabular
🎯 What it does: The study evaluates whether a minority of elected representatives can represent the will of the overall population in random elections (sortition) and provides theoretical limits on panel size.
Can AI Assistants Know What They Don't Know?
Qinyuan Cheng (Fudan University), Xipeng Qiu (Fudan University)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: This paper studies whether AI assistants can identify and express their knowledge boundaries, specifically by proactively stating 'I don't know' when they do not know the answer, and improving their accuracy and self-awareness through alignment methods.
Can Gaussian Sketching Converge Faster on a Preconditioned Landscape?
Yilong Wang (Xi'an Jiaotong University), Ivor Tsang
Optimization
🎯 What it does: This paper studies a new high-dimensional optimization algorithm called Gaussian Sketched Gradient Descent (GSGD), which achieves linear convergence under the condition of no important sampling through Gaussian sampling and adaptive gradient estimation, and can handle non-separable regularization terms.
Can Implicit Bias Imply Adversarial Robustness?
Hancheng Min (University of Pennsylvania), Rene Vidal
OptimizationAdversarial AttackImage
🎯 What it does: This study investigates the implicit bias of gradient flow in shallow networks and demonstrates that for high-dimensional data with small inter-class correlations, ReLU networks, although generalizing well after training, are not robust to small perturbations. A polynomial ReLU (pReLU) activation function is proposed, which is shown to align neurons with the center of each subclass, thereby achieving robustness to larger perturbations, along with theoretical analysis and experimental validation.
Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?
Khashayar Gatmiry (Massachusetts Institute of Technology), Sanjiv Kumar (Google Research)
OptimizationTransformerTabularOrdinary Differential Equation
🎯 What it does: The study investigates whether the cyclic Transformer can achieve multi-step gradient descent in in-context learning, providing theoretical proofs for global optimality and gradient flow convergence.
Can Machines Learn the True Probabilities?
Jinsook Kim (Underwood International College Yonsei University)
🎯 What it does: This paper formalizes the definition of machine learning and objective probability, establishes the necessary and sufficient conditions for whether machines can learn the true objective probability, and proves that if the true probability cannot be directly observed, then no machine can learn it.
Can Mamba Learn How To Learn? A Comparative Study on In-Context Learning Tasks
Jongho Park (KRAFTON), Dimitris Papailiopoulos (University of Wisconsin-Madison)
RetrievalOptimizationMeta LearningTransformerTabularChain-of-Thought
🎯 What it does: This paper systematically evaluates the contextual learning ability of non-attention state space models (especially Mamba) in various artificially constructed reasoning tasks and compares it with Transformers. It then proposes the MambaFormer hybrid model that integrates both architectures, significantly improving multi-task performance.
Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
Wu Lin (Vector Institute), Alireza Makhzani (University of Toronto)
OptimizationConvolutional Neural NetworkRecurrent Neural NetworkGraph Neural NetworkTransformerImageSequential
🎯 What it does: Propose to remove the square root from the adaptive gradient method and provide theoretical and empirical analysis.
Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data
Jiahan Zhang (Singapore University of Technology and Design), Lei Feng (Singapore University of Technology and Design)
ClassificationRecognitionTransformerPrompt EngineeringVision Language ModelImageMultimodality
🎯 What it does: A Candidate Pseudolabel Learning (CPL) framework is proposed, which enhances the performance of visual-language models on downstream tasks by generating candidate pseudolabels from unlabeled data and performing prompt fine-tuning.
CaPS: Collaborative and Private Synthetic Data Generation from Distributed Sources
Sikha Pentyala (University of Washington), Martine De Cock (Ghent University)
Data SynthesisSafty and PrivacyTabular
🎯 What it does: A framework named CaPS is proposed, which enables collaborative generation of differentially private synthetic tabular data from multi-party data sources with arbitrary distributions (horizontal, vertical, or mixed), avoiding the use of a trusted aggregator and ensuring input privacy.
CarbonNovo: Joint Design of Protein Structure and Sequence Using a Unified Energy-based Model
Milong Ren (Institute of Computing Technology, Chinese Academy of Sciences), Haicang Zhang (Institute of Computing Technology, Chinese Academy of Sciences)
Protein Structure PredictionDiffusion modelScore-based ModelSequentialBiomedical Data
🎯 What it does: A unified energy-based model called CarbonNovo has been developed, capable of simultaneously generating the structure and amino acid sequence of proteins, avoiding the structural noise and sequence overfitting issues associated with traditional two-stage design.
Careful with that Scalpel: Improving Gradient Surgery with an EMA
Yu-Guan Hsieh (Apple), Pierre Ablin (Apple)
OptimizationTransformerSupervised Fine-TuningImageText
🎯 What it does: This paper proposes a new gradient surgery method called Bloop, which uses orthogonal projection and Exponential Moving Average (EMA) to achieve a good balance between the main loss and the auxiliary loss.
CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Process
Guangyi Chen (Mohamed bin Zayed University of Artificial Intelligence), Kun Zhang (Carnegie Mellon University)
GenerationRepresentation LearningRecurrent Neural NetworkAuto EncoderVideoSequential
🎯 What it does: Proposes the CaRiNG method, which learns identifiable causal representations in non-invertible generative processes using temporal context.
CARTE: Pretraining and Transfer for Tabular Learning
Myung Jun Kim (Inria Saclay), Gael Varoquaux (Probabl.ai)
ClassificationOptimizationRepresentation LearningGraph Neural NetworkTransformerContrastive LearningTabular
🎯 What it does: This paper proposes a pre-training framework based on Graph Attention Networks, called CARTE, which enables transfer learning on table data without schema or entity alignment.
Cascade-CLIP: Cascaded Vision-Language Embeddings Alignment for Zero-Shot Semantic Segmentation
Yunheng Li (Nankai University), Ming-Ming Cheng (Nankai University)
SegmentationTransformerVision Language ModelContrastive LearningImage
🎯 What it does: A Cascade-CLIP framework is designed, which splits the CLIP visual encoder into multiple stages and equips each stage with an independent text-image decoder, achieving cascading alignment of multi-level visual features and text embeddings, thereby enhancing zero-shot semantic segmentation performance.
CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modelling
Junchao Gong (Shanghai Jiao Tong University), Wanli Ouyang (Shanghai AI Laboratory)
GenerationData SynthesisComputational EfficiencyTransformerDiffusion modelAuto EncoderTime Series
🎯 What it does: This paper proposes CasCast, which utilizes a cascade framework to separate large-scale forecasting from small-scale generation, improving the accuracy of short-term radar precipitation forecasts, especially for extreme precipitation.
Case-Based or Rule-Based: How Do Transformers Do the Math?
Yi Hu (Peking University), Muhan Zhang (Peking University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The study investigates whether the Transformer relies on cases or rules when performing arithmetic reasoning, using intervention experiments to verify that it is case-based reasoning, and proposes Rule-Following Fine-Tuning (RFFT) to enable the model to reason step-by-step according to rules.
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning
Libin Zhu (University of California San Diego), Mikhail Belkin (University of California San Diego)
OptimizationRepresentation LearningConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: By analyzing the loss spikes during the SGD training process, the author explains them as catapult dynamics related to high learning rates and demonstrates that these spikes mainly occur in the principal feature subspace of the Neural Tangent Kernel (NTK).
Category-Aware Active Domain Adaptation
Wenxiao Xiao (Brandeis University), Hongfu Liu (Brandeis University)
Domain AdaptationImageBenchmark
🎯 What it does: This paper proposes a single-class active domain adaptation method that actively queries and labels the most beneficial samples in the target domain to improve the performance of a specified class, addressing the issue of existing active domain adaptation methods overly focusing on easily classifiable categories while neglecting difficult-to-classify ones.
CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables
Jiecheng Lu (Georgia Institute of Technology), Shihao Yang (Georgia Institute of Technology)
Time Series
🎯 What it does: The CATS framework is proposed and implemented, which captures cross-series relationships in multivariate time series by constructing Auxiliary Time Series (ATS) and generates final prediction results through simple predictor fusion.
CauDiTS: Causal Disentangled Domain Adaptation of Multivariate Time Series
junxin lu, Shiliang Sun (Shanghai Jiao Tong University)
ClassificationDomain AdaptationRecurrent Neural NetworkGraph Neural NetworkContrastive LearningTime Series
🎯 What it does: This study investigates multivariate time series classification under unsupervised domain adaptation, achieving more robust representations by learning domain-invariant causal reasoning and domain-specific correlations through causal decomposition.
Causal Action Influence Aware Counterfactual Data Augmentation
Núria Armengol Urpí (ETH Zurich), Georg Martius (University of Tübingen)
Robotic IntelligenceReinforcement Learning from Human FeedbackTabular
🎯 What it does: This paper proposes an offline data augmentation method based on causal action influence, CAIAC, which utilizes collected data to generate feasible counterfactual transformations, enhancing the generalization ability of robot learning.
Causal Bandits: The Pareto Optimal Frontier of Adaptivity, a Reduction to Linear Bandits, and Limitations around Unknown Marginals
Ziyi Liu (University of Toronto), Daniel M. Roy (University of Toronto)
Reinforcement Learning
🎯 What it does: This paper studies how to simultaneously achieve optimal regret minimization in both known and unknown causal structures within the causal bandit framework.
Causal Customer Churn Analysis with Low-rank Tensor Block Hazard Model
Chenyin Gao (North Carolina State University), Shu Yang (North Carolina State University)
Recommendation SystemOptimizationSupervised Fine-TuningTabularFinance Related
🎯 What it does: This paper proposes a low-rank tensor block hazard model for causal customer churn analysis under multiple interventions; it estimates the potential churn probability through 1-bit tensor completion and clusters the interventions.
Causal Discovery via Conditional Independence Testing with Proxy Variables
Mingzhou Liu (Peking University), Yizhou Wang (Peking University)
Biomedical DataElectronic Health Records
🎯 What it does: A non-parametric hypothesis testing method based on proxy variables is proposed, which tests causal relationships between continuous variables using discretization techniques, avoiding biases caused by unobserved confounding in traditional conditional independence tests.
Causal Discovery with Fewer Conditional Independence Tests
Kirankumar Shiragur (Broad Institute), Caroline Uhler (Massachusetts Institute of Technology)
Graph Neural NetworkGraphTabular
🎯 What it does: A causal graph representation that can be recovered with only polynomially many conditional independence tests is proposed—Causal Consistent Partition Graph (CCPG), along with an efficient corresponding algorithm; under identifiable graphs or situations with sufficient interventions, a complete DAG can be further recovered.
Causal Effect Identification in LiNGAM Models with Latent Confounders
Daniele Tramontano (Technical University of Munich), Negar Kiyavash (Ecole Polytechnique Federale de Lausanne)
Graph
🎯 What it does: The study provides general identifiability conditions for causal effects between observable variables in linear non-Gaussian causal models (LiNGAM) that include potential confounding variables, and proposes corresponding judgment algorithms and estimation methods.
Causal Inference from Competing Treatments
Ana-Andreea Stoica (Max Planck Institute for Intelligent Systems), Moritz Hardt (Max Planck Institute for Intelligent Systems)
🎯 What it does: This paper constructs a game-theoretic model aimed at estimating causal effects in competitive environments, particularly in situations where multiple treatment administrators (such as online advertisements) compete for the attention of subjects.
Causal Inference out of Control: Estimating Performativity without Treatment Randomization
Gary Cheng (Stanford University), Celestine Mendler-Dünner (Max Planck Institute for Intelligent Systems)
Recommendation SystemTime Series
🎯 What it does: The study investigates how to estimate the causal effects of digital platform algorithms on user consumption (i.e., performativity) using observational data without randomizing platform operations.
Causal Representation Learning from Multiple Distributions: A General Setting
Kun Zhang (Carnegie Mellon University), Yujia Zheng (Carnegie Mellon University)
Representation LearningFlow-based ModelAuto EncoderTime Series
🎯 What it does: The study focuses on multi-distribution (heterogeneous data or non-stationary time series) and utilizes non-parametric nonlinear mixed functions and latent causal structures to achieve the identifiability of latent causal variables and the recovery of causal graphs.
Causal Representation Learning Made Identifiable by Grouping of Observational Variables
Hiroshi Morioka (RIKEN Center for Advanced Intelligence Project), Aapo Hyvarinen (University of Helsinki)
Representation LearningContrastive LearningGraphTabular
🎯 What it does: A causal representation learning (CRL) model is proposed that utilizes the grouping structure of observed variables to achieve identifiability, along with a corresponding self-supervised learning framework G-CaRL;
Causal-IQA: Towards the Generalization of Image Quality Assessment Based on Causal Inference
Yan Zhong (Peking University), Tingting Jiang (Peking University)
Meta LearningImage
🎯 What it does: This paper proposes a no-reference image quality assessment (BIQA) method based on causal reasoning, Causal-IQA, which utilizes causal graphs and backdoor adjustment to eliminate the confounding effects of distortion types and image content, thereby enhancing the model's generalization ability.
Causality Based Front-door Defense Against Backdoor Attack on Language Models
Yiran Liu (Tsinghua University), Yang Yu (Tsinghua University)
ClassificationAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A front-door adjustment defense framework (FABE) based on causal inference has been designed and implemented to counter backdoor attacks on language models.
Causally Motivated Personalized Federated Invariant Learning with Shortcut-Averse Information-Theoretic Regularization
Xueyang Tang (Hong Kong Polytechnic University), Yue Yu (Peng Cheng Laboratory)
Domain AdaptationFederated LearningContrastive LearningImage
🎯 What it does: A personalized federated learning framework FedPIN based on causal structure is proposed, utilizing information-theoretic regularization to achieve shortcut-avoiding features and enhance out-of-domain (OOD) generalization capability.
CCM: Real-Time Controllable Visual Content Creation Using Text-to-Image Consistency Models
Jie Xiao (University of Science and Technology of China), Zheng-Jun Zha (University of Science and Technology of China)
GenerationData SynthesisSuper ResolutionDiffusion modelImageOrdinary Differential Equation
🎯 What it does: This paper studies how to achieve real-time controllable visual content generation on consistency models, and proposes to directly learn ControlNet through consistency training and design a unified adapter to bridge the gap between the Diffusion model ControlNet and consistency models.
Cell2Sentence: Teaching Large Language Models the Language of Biology
Daniel Levine (Yale University), David van Dijk (Yale University)
ClassificationGenerationTransformerLarge Language ModelSupervised Fine-TuningBiomedical Data
🎯 What it does: Transform the gene expression matrix from single-cell RNA-seq into sentences of gene names sorted in descending order of expression (Cell2Sentence), and fine-tune large language models (such as GPT-2, Pythia) using these sentences to achieve cell generation, cell type prediction, and natural language summary generation based on cell sentences.
Centralized Selection with Preferences in the Presence of Biases
L. Elisa Celis (Yale University), Andrew Xu (Yale University)
Tabular
🎯 What it does: The study investigates how to maximize utility while considering candidate preference fairness and group representativeness fairness in centralized selection problems with systemic bias.
Certifiably Byzantine-Robust Federated Conformal Prediction
Mintong Kang (University of Illinois), Bo Li (University of Chicago)
Federated LearningImageTabular
🎯 What it does: A Rob-FCP algorithm is proposed for achieving reliable distributed uncertainty quantification and prediction set construction in a federated learning environment with Byzantine attacks.
CF-OPT: Counterfactual Explanations for Structured Prediction
Germain Vivier-Ardisson (Polytechnique Montreal), Thibaut Vidal (Polytechnique Montreal)
OptimizationExplainability and InterpretabilityAuto EncoderImageTabular
🎯 What it does: The paper proposes a counterfactual explanation method for structured prediction pipelines, enhancing model transparency through latent space search.
CHAI: Clustered Head Attention for Efficient LLM Inference
Saurabh Agarwal (University of Wisconsin-Madison), Carole-Jean Wu (Meta)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper studies a method called CHAI, which is a dynamic inference clustering attention head approach aimed at reducing the computational and memory consumption of large language models (LLMs).
Chain of Code: Reasoning with a Language Model-Augmented Code Emulator
Chengshu Li (Stanford University), brian ichter
Robotic IntelligenceAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposes the Chain of Code (CoC) method, which utilizes language models to generate executable code, and when the code is not executable, simulates execution using the language model (LMulator), achieving a mixed reasoning process of 'code + reasoning'.
Chain-of-Thought Predictive Control
Zhiwei Jia (University of California San Diego), Hao Su (University of California San Diego)
OptimizationRobotic IntelligenceTransformerPrompt EngineeringPoint CloudChain-of-Thought
🎯 What it does: A hierarchical imitation learning method called Chain-of-Thought Predictive Control (CoTPC) is proposed for learning low-level control from suboptimal demonstrations, along with an unsupervised sub-skill decomposition and dynamic sub-skill guidance mechanism.
Challenges and Considerations in the Evaluation of Bayesian Causal Discovery
Amir Mohammad Karimi Mamaghan (KTH Royal Institute of Technology), Stefan Bauer (TU Munich)
Tabular
🎯 What it does: This study systematically evaluates the assessment metrics of Bayesian Causal Discovery (BCD) methods, demonstrating that commonly used proxy metrics (such as E-SHD, AUPRC, NLL, etc.) have a poor relationship with the true posterior in high-entropy scenarios with small sample sizes and non-identifiable models. It proposes two alternative downstream evaluation strategies: experimental design and causal effect estimation.
Challenges in Training PINNs: A Loss Landscape Perspective
Pratik Rathore (Stanford University), Madeleine Udell (Stanford University)
OptimizationComputational EfficiencyPhysics RelatedOrdinary Differential Equation
🎯 What it does: Research and optimize the training process of Physics-Informed Neural Networks (PINNs), focusing on the condition number issue of the loss landscape;
Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scale
Candi Zheng (Hong Kong University of Science and Technology), Yuan Lan (Southern University of Science and Technology)
GenerationData SynthesisDiffusion modelImagePhysics Related
🎯 What it does: A feature-guided method is proposed to correct the nonlinear errors of diffusion models under a high guidance ratio.
Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation
Randall Balestriero (Brown University), Sarath Shekkizhar (Tenyx)
GenerationAdversarial AttackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Analyzing the internal mechanisms of LLM from a geometric perspective, deriving the intrinsic dimensions of multi-head attention embeddings and MLP partition mappings, and proposing methods for toxicity detection and bypassing RLHF attacks using these geometric features.
Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum
Tin Sum Cheng (University of Basel), David Belius (UniDistance Suisse)
Tabular
🎯 What it does: This study investigates the overfitting behavior of the kernel ridge regression (KRR) ridgeless model under sub-Gaussian design assumptions in the context of different spectral decay (polynomial and exponential) and provides corresponding non-asymptotic error upper bounds and condition number analysis.
Characterizing ResNet's Universal Approximation Capability
Chenghao Liu (City University of Hong Kong), Minghua Chen (City University of Hong Kong)
Convolutional Neural Network
🎯 What it does: The research proves the universal approximation capability of ResNet with constant width in approximating polynomials, smooth functions, and Lebesgue integrable functions, providing both upper and lower bounds.
Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension
Fan Yin (University of California), Kai-Wei Chang (University of California)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The study investigates how to utilize the local intrinsic dimension (LID) of the internal representations of large language models to discern the truthfulness of generated text (to prevent hallucinations).
Chasing Convex Functions with Long-term Constraints
Adam Lechowicz (University of Massachusetts Amherst), Prashant Shenoy (University of Massachusetts Amherst)
OptimizationTabular
🎯 What it does: This paper proposes a new online convex function tracking problem—Convex Function Tracking with Long-term Constraints (CFL) and its special case in weighted ℓ1 space (MAL), and designs optimal competitive algorithm ALG1 and learning-enhanced algorithm CLIP;
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Wei-Lin Chiang (University of California Berkeley), Ion Stoica (University of California Berkeley)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelText
🎯 What it does: An open, human-preference-based real-time large language model (LLM) evaluation platform called Chatbot Arena has been constructed, and approximately 100K pairs of model comparison preference data have been made public.
CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space using Quantum-Chemical Feedback
Henry W. Sprueill (Pacific Northwest National Laboratory), Sutanay Choudhury (Pacific Northwest National Laboratory)
Drug DiscoveryGraph Neural NetworkTransformerLarge Language ModelReinforcement LearningGraphChain-of-Thought
🎯 What it does: A heuristic search framework named CHEMREASONER has been developed, combining large language models (LLM) with quantum chemistry (GNN) feedback to achieve automated catalyst discovery.
CKGConv: General Graph Convolution with Continuous Kernels
Liheng Ma (McGill University), Mark Coates (McGill University)
Graph Neural NetworkGraphBenchmark
🎯 What it does: A general continuous kernel graph convolution framework (CKGConv) is proposed, which generates pseudo-coordinates using graph positional encoding and learns convolution kernels in the form of continuous functions.
Class-Imbalanced Graph Learning without Class Rebalancing
Zhining Liu (University of Illinois Urbana Champaign), Hanghang Tong (University of Illinois Urbana Champaign)
ClassificationGraph Neural NetworkGraph
🎯 What it does: This paper proposes a category-free rebalancing method based on topological enhancement called BAT (BAlanced Topological Augmentation), which alleviates the class imbalance bias in graph learning by dynamically locating and correcting nodes affected by AMP (Ambivalent Message-Passing) and DMP (Distant Message-Passing).
Classification under Nuisance Parameters and Generalized Label Shift in Likelihood-Free Inference
Luca Masserano (Carnegie Mellon University), Ann B. Lee (Carnegie Mellon University)
ClassificationBiomedical Data
🎯 What it does: In the presence of noise parameters and label distribution shift (GLS), a reliable classification and uncertainty quantification method is proposed, which can provide a prediction set that satisfies conditional and marginal coverage without retraining the model;
Classification Under Strategic Self-Selection
Guy Horowitz (Technion Israel Institute of Technology), Nir Rosenfeld (Technion Israel Institute of Technology)
ClassificationOptimizationTabular
🎯 What it does: The study examines the self-selection behavior of candidates based on whether they apply for a learned classifier in selective machine learning, and proposes a differentiable learning framework to simultaneously optimize prediction accuracy and self-selection distribution.
CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networks
Yulong Huang (Hong Kong University of Science and Technology), Bojun Cheng (Hong Kong University of Science and Technology)
Spiking Neural NetworkImage
🎯 What it does: A Complementary Leaky Integrate-and-Fire (CLIF) neuron is proposed and implemented to address the vanishing gradient problem encountered by traditional LIF neurons during SNN training, allowing for direct replacement of LIF in training across various network architectures.
Clifford-Steerable Convolutional Neural Networks
Maksim Zhdanov (University of Amsterdam), Patrick Forré (University of Amsterdam)
Convolutional Neural NetworkTime SeriesSequentialPhysics Related
🎯 What it does: This paper proposes the Clifford-Steerable CNN (CS-CNN) framework, which is capable of maintaining equivariance in convolutional neural networks under isometric transformations E(p,q) in pseudo-Euclidean space R^{p,q}, handling multi-vector fields.
CLIPZyme: Reaction-Conditioned Virtual Screening of Enzymes
Peter Mikhael, Regina Barzilay (Massachusetts Institute of Technology)
Drug DiscoveryProtein Structure PredictionGraph Neural NetworkContrastive LearningGraph
🎯 What it does: Developed CLIPZyme, a virtual enzyme screening framework based on contrastive learning, which ranks enzyme sequences given a chemical reaction to identify the catalytic activity of natural enzymes.
CLLMs: Consistency Large Language Models
Siqi Kou (Shanghai Jiao Tong University), Hao Zhang (University of California)
GenerationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: In response to the sequential bottleneck of LLM inference, a new Consistency Large Language Models (CLLMs) is proposed, which fine-tunes the pre-trained model to predict multiple extra tokens at once during the Jacobi parallel decoding process, thereby achieving efficient parallel inference.
Closing the Gap: Achieving Global Convergence (Last Iterate) of Actor-Critic under Markovian Sampling with Neural Network Parametrization
Mudit Gaur (Purdue University), Vaneet Aggarwal (Purdue University)
OptimizationReinforcement Learning
🎯 What it does: This paper proposes a parameterized Actor-Critic algorithm for multi-layer neural networks, providing a global convergence sample complexity analysis across five practical dimensions: continuous state-action space, Markov sampling, and the 'last iteration' with global optimality, proving that the convergence rate is ~O(ε⁻³);
Cluster-Aware Similarity Diffusion for Instance Retrieval
Jifei Luo (University of Science and Technology of China), Changsheng Xu (Institute of Automation Chinese Academy of Sciences)
RetrievalImage
🎯 What it does: A clustering-aware similarity diffusion (CAS) algorithm is developed for re-ranking in instance retrieval.
Clustered Federated Learning via Gradient-based Partitioning
Heasung Kim (University of Texas at Austin), Gustavo De Veciana
OptimizationFederated LearningConvolutional Neural NetworkAuto EncoderImageTabular
🎯 What it does: This paper proposes a clustering-based group federated learning algorithm, CFL-GP, which can automatically partition clients into several clusters and train a shared model for each cluster without sharing the original data.
Coactive Learning for Large Language Models using Implicit User Feedback
Aaron David Tucker, Thorsten Joachims (Cornell University)
GenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningContrastive LearningText
🎯 What it does: This paper proposes a Coactive Learning-based LLM training framework called CoRLL, which utilizes users' spontaneous text edits during usage as implicit preference feedback.
COALA: A Practical and Vision-Centric Federated Learning Platform
Weiming Zhuang (Sony AI), Lingjuan Lyu (Sony AI)
Autonomous DrivingFederated LearningImage
🎯 What it does: A COALA platform has been constructed, providing a federated learning framework for visual tasks and multi-scenario benchmarks.
Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models
Qitan Lv (University of Science and Technology of China), Feng Wu (University of Science and Technology of China)
TransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: A COFT framework is proposed, which reduces the knowledge hallucination of large language models through a coarse-to-fine emphasis method.
Coarse-To-Fine Tensor Trains for Compact Visual Representations
Sebastian Bugge Loeschcke (University of Copenhagen), Sagie Benaim (Hebrew University of Jerusalem)
RestorationCompressionNeural Radiance FieldImagePoint Cloud
🎯 What it does: A method called Progressive Tensor Training (PuTT) is proposed for learning compact and high-quality representations of visual data.