International Conference on Learning Representations Β· 1682 papers
Context-Alignment: Activating and Enhancing LLMs Capabilities in Time Series
Yuxiao Hu (Hong Kong Polytechnic University), Yuntian Chen (Ningbo Institute of Digital Twin)
CodeClassificationGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTime Series
π― What it does: Proposes the Context-Alignment concept, utilizing a Dual-Scale GNN to achieve structural and logical alignment between time series and language prompts, activating and enhancing the performance of large language models in temporal tasks.
ContextGNN: Beyond Two-Tower Recommendation Systems
Yiwen Yuan (Kumo.AI), Matthias Fey (Kumo.AI)
CodeRecommendation SystemGraph Neural NetworkTime Series
π― What it does: A Context-based Graph Neural Network that integrates pair-wise and two-tower representations is proposed for time series recommendation.
Contextualizing biological perturbation experiments through language
Menghua Wu (Massachusetts Institute of Technology), Jan-Christian Huetter (Genentech)
CodeTransformerLarge Language ModelBiomedical DataBenchmarkRetrieval-Augmented GenerationChain-of-Thought
π― What it does: This paper proposes the PERTURBQA benchmark to evaluate the ability of language models in reasoning about biological perturbation experimental results, and constructs the SUMMER framework to combine knowledge graphs with experimental data for question-answer reasoning.
π― What it does: This paper proposes CoSFan, which combines fast 'what-how' inference through continual meta-learning with slow 'when' adaptation to achieve continual learning in high-dimensional time series prediction with unknown task boundaries and identities.
π― What it does: This paper proposes CDTD, a continuous diffusion model for mixed tabular data, which uniformly uses Gaussian noise to diffuse continuous features and embedded categorical features, combining score matching and score interpolation.
π― What it does: A continuous ensemble forecasting framework based on diffusion models is proposed, which directly models the distribution of arbitrary time delays in the noise space of the diffusion process and generates time-continuous trajectories using correlated noise; at the same time, it combines autoregressive continuous interpolation (ARCI) to achieve compatibility between long time delay forecasting and high temporal resolution.
π― What it does: A continuous exposure learning method CLODE based on Neural Ordinary Differential Equations (NODE) is proposed, improving the convergence and effectiveness of traditional discrete iterative curve adjustment in unsupervised low-light image enhancement.
π― What it does: The ContraDiff method is proposed, which utilizes contrastive learning to treat low-reward trajectories from offline data as negative samples and high-reward trajectories as positive samples. By applying contrastive constraints to the states of generated trajectories, it enhances the performance of offline RL when high-reward samples are scarce.
π― What it does: By applying Gaussian noise perturbations to the parameters of a sound synthesizer, similar but not identical audio pairs (audio doppelgangers) are generated and used as positive samples for contrastive learning, thereby obtaining robust audio representations.
π― What it does: A controllable autoregressive image generation framework called ControlAR is proposed, which embeds spatial control information into the autoregressive model through conditional decoding, achieving high-quality image generation at arbitrary resolutions.
Controlled LLM Decoding via Discrete Auto-regressive Biasing
Patrick Pynadath (Purdue University), Ruqi Zhang (Purdue University)
CodeGenerationTransformerLarge Language ModelText
π― What it does: A controlled decoding algorithm DAB based on discrete gradient sampling is proposed to generate fluent text while satisfying external constraints.
Controlling Language and Diffusion Models by Transporting Activations
Pau Rodriguez, Xavier Suau (Apple)
CodeGenerationData SynthesisOptimizationTransformerLarge Language ModelDiffusion modelImageTextMultimodality
π― What it does: This paper proposes the Activation Transport (ACT) framework, which achieves fine-grained controllability of large generative models (LLM and T2I) by optimally transporting activation values during inference.
Convex Formulations for Training Two-Layer ReLU Neural Networks
Karthik Prakhya (UmeΓ₯ University), Alp Yurtsever (Imperial College London)
CodeOptimizationTabularSequential
π― What it does: Transform the training problem of two-layer ReLU networks into a convex copositive program (completely positive program), and based on this, propose a solvable semidefinite relaxation and rounding method;
Copyright-Protected Language Generation via Adaptive Model Fusion
Javier Abad (ETH Zurich), Fanny Yang (ETH Zurich)
CodeGenerationOptimizationTransformerLarge Language ModelText
π― What it does: A post-processing model fusion method named CP-Fuse is proposed to reduce the risk of language models generating copyrighted content during the inference phase by combining two models trained on non-overlapping copyrighted data.
π― What it does: This paper proposes a core subset selection method based on Reducible Loss (ReL) called CSReL, specifically designed to enhance the quality of sample memory in experience replay (ER) based continuous learning (CL). Additionally, three extension schemes are provided, considering task interference, streaming data, and knowledge distillation (CSReL-CL, CSReL-RS, CSReL-RS-KD).
CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking
Tarun Suresh (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
CodeRetrievalAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
π― What it does: A high-quality contrastive learning dataset called CORNSTACK was constructed, and this dataset was used to train a code retriever and re-ranker, significantly improving the performance of code retrieval and functionality localization.
Correlated Proxies: A New Definition and Improved Mitigation for Reward Hacking
Cassidy Laidlaw (University of California), Anca Dragan (University of California)
CodeOptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabularTime Series
π― What it does: A new definition of reward hacking is proposed, and theoretical and empirical protection against reward hacking is achieved through ΟΒ² occupancy measure regularization.
Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)
SUBBA REDDY OOTA, Manish Gupta (Microsoft)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageMultimodalityMagnetic Resonance Imaging
π― What it does: This paper utilizes instruction-tuned multimodal large language models (MLLMs) to extract instruction-specific text output embeddings in natural image viewing tasks and predicts human fMRI brain activity using a linear encoding model, studying the effects of different instructions, levels, and visual concepts on brain encoding.
Correlation and Navigation in the Vocabulary Key Representation Space of Language Models
Letian Peng (University of California), Jingbo Shang (University of California)
CodeGenerationKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper studies the impact of irrelevant correlations in the key space of the vocabulary on next word prediction and proposes an in-context navigation (ICN) method that iteratively pushes the query vector away from explored keys by incorporating previously explored answers into the context, thereby alleviating this correlation.
Gabriele Dominici (UniversitΓ della Svizzera italiana), Marc Langheinrich (UniversitΓ della Svizzera italiana)
CodeExplainability and InterpretabilityAdversarial AttackAuto EncoderGenerative Adversarial NetworkImage
π― What it does: A joint Generative Adversarial Concept Bottleneck Model (CF-CBM) is proposed, addressing three core issues: prediction, scenario simulation, and adversarial explanation.
π― What it does: Proposes Consistency-Regularized CTC (CR-CTC), which utilizes the CTC distribution of two different augmented views of the same speech for consistency regularization to enhance the recognition performance of pure CTC.
π― What it does: This paper proposes a Continuous Relative Rotation Position Query (CR2PQ) framework to eliminate the need for pixel/patch correspondence in dense visual representation learning and achieve cross-view alignment through a relative coordinate system and continuous RoPE.
CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair
Mingjie Liu (NVIDIA Corporation), Haoxing Ren (NVIDIA Corporation)
CodeData SynthesisAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: A high-quality synthesis data generation framework for Verilog code generation has been developed, which includes correctly constructed non-text representations (Karnaugh maps, FSMs, waveforms) and automated target code repair data, followed by fine-tuning on Starcoder2-15B.
CREAM: Consistency Regularized Self-Rewarding Language Models
Zhaoyang Wang (University of North Carolina at Chapel Hill), Huaxiu Yao (University of North Carolina at Chapel Hill)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: This paper proposes a self-rewarding language model based on consistency regularization (CREAM), which reduces reward bias by utilizing reward consistency between different iterations during the self-rewarding process, thereby enhancing the alignment performance of a small 7B LLM.
π― What it does: Introducing task credit-based self-organizing maps (CB-SOM) in deep convolutional networks allows the filters of all layers to form a topological structure while maintaining low task performance loss.
Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion
Marco Mistretta (University of Florence), Andrew D. Bagdanov (University of Florence)
CodeRetrievalTransformerVision Language ModelContrastive LearningImageText
π― What it does: This study investigates the intra-modal mismatch problem present in single-modal tasks (image-image retrieval, text-text retrieval) using the CLIP pre-trained model. It transforms single-modal tasks into cross-modal tasks through single-feature-level modality inversion (OTI and OVI) to leverage the cross-modal alignment advantages of CLIP and improve retrieval performance.
Cross-Attention Head Position Patterns Can Align with Human Visual Concepts in Text-to-Image Generative Models
Jungwon Park (Seoul National University), Wonjong Rhee (Seoul National University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageText
π― What it does: This paper proposes a Head Relevant Vector (HRV) method based on cross-attention heads to map human-specified visual concepts to the attention heads of stable diffusion models, enhancing and adjusting concepts in image generation, editing, and multi-concept generation tasks.
π― What it does: Explores cross-domain offline reinforcement learning in the context of limited target domain data, using source domain data to assist in target domain policy learning.
π― What it does: The study investigates whether models trained with cross-entropy classification can reverse the data generation process and provides a theory of identifiability.
Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models
Shicheng Xu (Chinese Academy of Sciences), Xueqi Cheng (Chinese Academy of Sciences)
CodeSafty and PrivacyTransformerVision Language ModelImageTextMultimodality
π― What it does: This paper presents a new perspective called cross-modal safety mechanism transfer, aimed at addressing the vulnerabilities of large visual language models (LVLMs) when handling toxic visual inputs. By comparing the safety mechanisms of text and visuals, it was found that existing methods are unable to effectively transfer the safety mechanisms of text to visuals.
CrossMPT: Cross-attention Message-passing Transformer for Error Correcting Codes
Seong-Joon Park (POSTECH), Jong-Seon No (Seoul National University)
CodeTransformer
π― What it does: A new Transformer architecture called CrossMPT is proposed for decoding error correction codes (ECC); this architecture updates magnitude and syndrome embeddings through cross-attention, forming an iterative update process similar to information transmission.
cryoSPHERE: Single-Particle HEterogeneous REconstruction from cryo EM
Gabriel Ducrocq (Linkoping University), Fredrik Lindsten (Linkoping University)
CodeProtein Structure PredictionAuto EncoderImage
π― What it does: A cryoSPHERE method based on variational autoencoders is proposed, which utilizes benchmark structures predicted by AlphaFold to learn how to segment protein chains into several segments and predict rigid transformations for each segment in each cryo-EM image, thereby recovering the continuous conformation of the protein at the single-particle level.
CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation
Yifeng Xu (Institute of Computing Technology, Chinese Academy of Sciences), Xilin Chen (Institute of Computing Technology, Chinese Academy of Sciences)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: This paper proposes a scalable, low-resource image-to-image control generation framework called CtrLoRA, which first trains a shared Base ControlNet and learns LoRA for different conditions, then quickly adapts to new conditions with a small amount of data.
CURIE: Evaluating LLMs on Multitask Scientific Long-Context Understanding and Reasoning
Hao Cui (Google), Subhashini Venugopalan (Google)
CodeTransformerLarge Language ModelPrompt EngineeringTextMultimodalityBenchmarkPhysics Related
π― What it does: The CURIE benchmark is proposed to evaluate the long-context understanding, reasoning, and information extraction capabilities of large language models across six scientific fields (materials science, theoretical condensed matter physics, quantum computing, geospatial analysis, biodiversity, and protein structure);
Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems
Guibin Zhang (Tongji University), Tianlong Chen (University of North Carolina at Chapel Hill)
CodeTransformerLarge Language ModelAgentic AIText
π― What it does: This paper proposes AgentPrune, which utilizes a trainable low-rank graph mask to prune the space-time communication graph in LLM-based multi-agent systems in a one-time manner, significantly reducing token consumption while maintaining or even improving performance.
Cut Your Losses in Large-Vocabulary Language Models
Erik Wijmans (Apple), Philipp Kraehenbuehl
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Proposes the Cut Cross-Entropy (CCE) scheme, significantly reducing the memory usage of the cross-entropy layer in training large vocabulary LLMs.
CViT: Continuous Vision Transformer for Operator Learning
Sifan Wang (Yale University), Paris Perdikaris (University of Pennsylvania)
CodeTransformerMeshGraphPhysics Related
π― What it does: This paper proposes the Continuous Vision Transformer (CViT), a neural operator that integrates visual Transformers with continuous coordinate embeddings to learn the input-output mapping of PDEs.
Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection
Chuhan ZHANG, Dong Zhang (Hong Kong University of Science and Technology)
CodeObject DetectionKnowledge DistillationTransformerVision Language ModelContrastive LearningImage
π― What it does: Designed and implemented the CCKT-Det framework, achieving open vocabulary object detection by aligning language priors with visual region features in a cyclic manner without using additional annotations or pseudo-labels.
π― What it does: We propose D-FINE, a real-time object detector that achieves high-precision localization by redefining the bounding box regression task of DETR.
DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life
Yu Ying Chiu (University of Washington), Yejin Choi (University of Washington)
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: A dataset called DAILYDILEMMAS is proposed and constructed, containing 1,360 moral dilemmas in daily life with binary choices, and the value conflicts in these dilemmas are annotated based on the theory of five major values.
DAMO: Decoding by Accumulating Activations Momentum for Mitigating Hallucinations in Vision-Language Models
Kaishen Wang (University of Maryland), Kaixiong Zhou (North Carolina State University)
CodeTransformerVision Language ModelMultimodality
π― What it does: A decoding method based on activation momentum, DAMO, is proposed, which utilizes visual information from early layers to correct the activations of later layers during inference, significantly reducing the hallucination outputs of large-scale visual language models (LVLMs).
DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models
Wenlong Deng (University of British Columbia), Christos Thrampoulidis (University of British Columbia)
CodeTransformerSupervised Fine-TuningText
π― What it does: This study improves the Delta-parameter pruning (DPP) technique by addressing the limitations of the original random Drop-Rescale (DARE) method. It proposes a tunable rescaling factor DAREx-q and incorporates AdamR regularization during the fine-tuning phase in the DAREx-L2 scheme, achieving extreme pruning (up to 99%) while maintaining nearly original performance.
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
Jiasheng Ye (Fudan University), Xipeng Qiu (Fudan University)
CodeOptimizationData-Centric LearningTransformerLarge Language ModelText
π― What it does: A data mixing law has been developed that can predict the impact of different training data ratios on language model performance without training the full model. By nesting this law with scaling laws for training steps and model size, it allows for the prediction of the final performance of large-scale models at various mixing ratios using small-scale experiments.
π― What it does: A data unlearning method for diffusion models called SISS is proposed, which can effectively delete the memory of specified training samples while maintaining model quality.
Data-centric Prediction Explanation via Kernelized Stein Discrepancy
Mahtab Sarvmaili (Dalhousie University), Ga Wu (Dalhousie University)
CodeExplainability and InterpretabilityData-Centric LearningConvolutional Neural NetworkImageBiomedical Data
π― What it does: This paper proposes an example-based prediction explanation method called HD-Explain, which utilizes kernelized Stein discrepancy (KSD) to accurately extract training samples that support a given test point from a trained model.
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Designed and trained a data manager named DataMan, which performs 14 quality assessments and 15 domain labels on pre-trained corpora. Subsequently, data sampled based on these labels is used for LLM pre-training, significantly improving the model's perplexity, contextual learning, and instruction-following performance.
π― What it does: A contrastive relationship gap based on contrastive pre-training models is proposed to verify dataset ownership, targeting black-box scenarios to detect potential dataset theft.
DCT-CryptoNets: Scaling Private Inference in the Frequency Domain
Arjun Roy (Purdue University), Kaushik Roy (Purdue University)
CodeSafty and PrivacyComputational EfficiencyImage
π― What it does: A framework for fully homomorphic encrypted neural network inference in the frequency domain (DCT) called DCT-CryptoNets is proposed to reduce the computational overhead of nonlinear activation and homomorphic guiding switches.
DeciMamba: Exploring the Length Extrapolation Potential of Mamba
Assaf Ben-Kish (Tel Aviv University), Raja Giryes (Tel Aviv University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper systematically analyzes the bottlenecks of the Mamba model in length-extrapolation through visualization, measurement, and experiments, finding that its effective receptive field (ERF) is limited by the length of the training sequence. It then proposes a dynamic pooling mechanism (DeciMamba) based on the importance scores of the 'delta_t' from the S6 layer, achieving context compression and expansion for long sequences, thereby significantly enhancing Mamba's long sequence inference capability without retraining.
Decision Information Meets Large Language Models: The Future of Explainable Operations Research
Yansen Zhang (City University of Hong Kong), Chen Ma (City University of Hong Kong)
CodeOptimizationExplainability and InterpretabilityTransformerLarge Language ModelTextTabularBenchmark
π― What it does: Proposed the Explainable Operations Research (EOR) framework, which utilizes large language models (LLM) to provide interpretable decisions and explanations for operational optimization problems, and introduces the concept of 'decision information' along with a bipartite graph-based quantification method;
π― What it does: A query-based object detection and segmentation framework called DECO is proposed, which achieves competitive detection performance on the COCO dataset.
Decoding Game: On Minimax Optimality of Heuristic Text Generation Strategies
Sijin Chen (Princeton University), Jason Matthew Klusowski
CodeGenerationOptimizationTransformerLarge Language ModelText
π― What it does: A two-player zero-sum game framework (Decoding Game) is proposed to study the theoretical optimality of decoding strategies in text generation.
π― What it does: A new parameter-efficient fine-tuning method called DeLoRA is proposed, which enhances robustness without significantly increasing computational costs by adding normalization and a learnable scale factor Ξ» to the low-rank matrices of LoRA, decoupling angle learning from adaptation strength.
π― What it does: This paper proposes a hierarchical method that first generates the layout of running script using text content and style references, and then generates online Chinese characters in accordance with the style character by character through a diffusion model, thus completing the generation of a complete written text line.
DEEM: Diffusion models serve as the eyes of large language models for image perception
Run Luo (Shenzhen Key Laboratory for High Performance Data Mining), Min Yang (Shenzhen Key Laboratory for High Performance Data Mining)
CodeGenerationData SynthesisRetrievalTransformerLarge Language ModelDiffusion modelImageTextMultimodality
π― What it does: The DEEM method is proposed, which provides self-supervised visual consistency regularization through a diffusion model to correct the semantic distribution of the visual encoder, thereby enhancing the robustness and accuracy of large language models in image perception, generation, and cross-modal interaction.
π― What it does: This paper proposes the Deep Compression Autoencoder (DC-AE), which accelerates the training and inference of high-resolution diffusion models by improving the spatial compression ratio.
π― What it does: This paper proposes the Deep Ξ±-KP, a deep Ξ±-stable kernel process, as the limit of infinitely wide Bayesian neural networks with infinitely variance weights, and provides its conditional Gaussian mixture representation and recursive kernel formula, based on which posterior inference and prediction are achieved.
Deep Kernel Relative Test for Machine-generated Text Detection
Yiliao Song (University of Adelaide), Feng Liu (University of Technology Sydney)
CodeClassificationGenerationTransformerLarge Language ModelText
π― What it does: A non-parametric machine-generated text detection method based on kernel relative testing, R-Detect, is proposed. It utilizes text vector representation and kernel maximum mean discrepancy (MMD) for statistical significance testing, achieving zero-shot detection.
π― What it does: Proposes the Deep Signature framework for efficiently describing large-scale protein dynamics trajectories and predicting functional properties.
π― What it does: This paper proposes DeeperForward, which extends the Forward-Forward training method to a 17-layer CNN and improves the learning of deep networks through average affinity and layer normalization.
DeepRTL: Bridging Verilog Understanding and Generation with a Unified Representation Model
Yi Liu (National Technology Innovation Center for EDA), Qiang Xu (National Technology Innovation Center for EDA)
CodeGenerationAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkChain-of-Thought
π― What it does: This paper presents DeepRTL, a unified representation model for understanding and generating Verilog code, fine-tuned through curriculum learning.
DELIFT: Data Efficient Language model Instruction Fine-Tuning
Ishika Agarwal (University of Illinois Urbana-Champaign), Marina Danilevsky (IBM Research)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper presents DELIFT, a unified data subset selection framework that efficiently selects the most valuable training samples during three stages: instruction tuning, task-specific fine-tuning, and continuous fine-tuning, significantly reducing the amount of data required.
DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory
Yutong Wang (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: An online document-level translation agent named DELTA has been developed, utilizing a multi-layer memory structure (proper noun records, bilingual summaries, long-term and short-term memory) combined with LLM for sentence-level translation, aiming to enhance the consistency and quality of document translation.
π― What it does: A defense framework called Democratic Training is proposed to resist Universal Adversarial Perturbation (UAP) attacks on deep networks, primarily by using low-entropy samples during the model fine-tuning process to 'weaken' the dominant features of UAP.
π― What it does: Analyzed the dynamic behavior of tokens in the pre-trained Mamba (Selective State Space Model), proving the existence of both convergence and divergence in the one-dimensional case, and exploring its impact on model performance.
π― What it does: This paper proposes the Dense Video Object Captioning task, which unifies detection, tracking, and generating coherent captions for all objects in a video.
Density estimation with LLMs: a geometric investigation of in-context learning trajectories
Toni J.B. Liu (Cornell University), Christopher Earls
CodeTransformerLarge Language ModelPrompt Engineering
π― What it does: Exploring the ability of large language models to estimate probability density functions in context learning and using InPCA to visualize their learning trajectories; interpreted as adaptive kernel density estimation.
π― What it does: This paper presents Depth Any Video, a foundational model for depth estimation that can simultaneously process images and videos, achieving high-quality, temporally consistent depth predictions on videos of arbitrary lengths.
Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
Alexey Bochkovskiy, Vladlen Koltun (Apple)
CodeDepth EstimationTransformerImage
π― What it does: A zero-shot multi-scale ViT base model called Depth Pro is proposed for generating high-resolution, absolute scale monocular depth maps without the need for camera intrinsics.
π― What it does: A concise convolutional network framework named CoSNet is proposed, which employs parallel column convolution, input replication, minimized 1Γ1 convolution, and single fusion designs, aiming to significantly reduce parameters, FLOPs, and inference latency while maintaining high accuracy.
Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling
Yuxuan Yao (City University of Hong Kong), Linqi Song (City University of Hong Kong)
CodeTransformerLarge Language ModelMixture of ExpertsTextBenchmark
π― What it does: This paper researches and implements an efficient LLM integration method (UNITE) and proposes a model selection strategy based on model performance and response style, enhancing multi-model inference performance.
π― What it does: This paper studies low-bit quantization for text-to-image diffusion models and proposes a method called Distribution-aware Group Quantization (DGQ), which maintains high-quality image and text alignment at below 8-bit precision without requiring additional fine-tuning.
DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image
Qingxuan Wu (University of Pennsylvania), Lingjie Liu (University of Pennsylvania)
CodePose EstimationTransformerImageMesh
π― What it does: This paper proposes an end-to-end single-image hand-face interaction 3D reconstruction method called DICE, which can recover the deformations in hand and face interactions.
π― What it does: This paper proposes a diffusion model-based prompt generator called Diff-Prompt, which generates rich and fine-grained input-specific prompts to assist in the efficient fine-tuning of pre-trained multimodal models.
Differentiable and Learnable Wireless Simulation with Geometric Transformers
Thomas Hehn (Qualcomm AI Research), Johann Brehmer (Qualcomm AI Research)
CodeTransformerDiffusion modelMesh
π― What it does: Designed and trained a fully learnable and differentiable wireless signal propagation simulator, Wi-GATr, which predicts radio frequency channel characteristics using 3D geometric meshes and antenna location information, and supports inverse localization and geometric reconstruction.
Differentiable Causal Discovery for Latent Hierarchical Causal Models
Parjanya Prajakta Prashant (University of California San Diego), Biwei Huang (University of California San Diego)
CodeAuto EncoderImageTabular
π― What it does: A differentiable causal discovery method is proposed to recover the structure of nonlinear latent hierarchical causal models from observational data, along with identifiable theory and implementation algorithms.
Zijie Geng (University of Science and Technology of China), Feng Wu (University of Science and Technology of China)
CodeOptimizationGraph Neural Network
π― What it does: This paper proposes DiffILO, a differentiable integer linear programming method based on unsupervised learning, aimed at directly predicting feasible solutions for ILP.
Differentiable Optimization of Similarity Scores Between Models and Brains
Nathan Cloos (Massachusetts Institute of Technology), Christopher J Cueva
CodeOptimizationBiomedical Data
π― What it does: This study investigates various commonly used model-brain similarity measures, revealing through differentiable gradient optimization that high scores do not necessarily represent the similarity of task-related information.
CodeOptimizationSafty and PrivacyLarge Language ModelPrompt EngineeringText
π― What it does: The research aligns the behavior of large language models using activation editing while maintaining differential privacy guarantees.
π― What it does: This paper proposes an unsupervised graph anomaly detection method based on diffusion models, called DiffGAD. It utilizes a graph autoencoder to map the graph to a latent space, and then employs a diffusion model to reconstruct and discriminate the latent representations, thereby achieving node-level anomaly detection.
DiffPuter: Empowering Diffusion Models for Missing Data Imputation
Hengrui Zhang (University of Illinois at Chicago), Philip S. Yu (University of Illinois at Chicago)
CodeDiffusion modelScore-based ModelTabular
π― What it does: An iterative missing value imputation method called DIFFPUTER is proposed, which combines the Expectation-Maximization (EM) algorithm and diffusion models to simultaneously estimate the complete data distribution and update missing values.
π― What it does: This paper proposes a diffusion model-based imitation learning framework called SMILING, which directly compares the state distributions of experts and learners using score matching, thereby avoiding the training of adversarial discriminators.
π― What it does: Utilizing a lightweight coarse-tuning search to collect sub-optimal model checkpoints, and then using a graph-conditioned latent diffusion model to generate better GNN parameters, thus achieving improved performance on various graph tasks with minimal hyperparameter tuning.
π― What it does: Proposes the Diffusion Actor-Critic (DAC) framework, which utilizes diffusion models to directly learn the target policy in offline reinforcement learning and alternately trains with the Critic to complete KL-constrained policy iteration.
π― What it does: This paper proposes the Diffusion Attribution Score (DAS) to evaluate the influence of each training sample on the generated results in diffusion models.
π― What it does: Proposes Diffusion Bridge AutoEncoders (DBAE) for unsupervised representation learning, addressing the information splitting problem in diffusion models.
Wenxuan Wang (Institute of Automation Chinese Academy of Sciences), Xinlong Wang (Beijing Academy of Artificial Intelligence)
CodeClassificationSegmentationRetrievalOptimizationRepresentation LearningTransformerVision Language ModelDiffusion modelContrastive LearningImageMultimodality
π― What it does: A post-training framework DIVA based on diffusion model feedback is proposed, which utilizes pure image data for self-supervised optimization of CLIP visual representations, significantly enhancing its fine-grained visual perception capabilities.
Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images
Sichen Zhu (Georgia Institute of Technology), Peng Qiu (Georgia Institute of Technology)
CodeGenerationDiffusion modelImageBiomedical Data
π― What it does: This paper presents Stem, a computational method based on conditional diffusion models, used to infer spatially resolved gene expression profiles from H&E tissue slice images.
π― What it does: The paper proves that diffusion models are essentially a type of evolutionary algorithm, and based on this, proposes two evolutionary optimization methods: Diffusion Evolution and Latent Space Diffusion Evolution, which can discover diverse and high-quality solutions in multimodal and high-dimensional parameter spaces.
π― What it does: This study investigates high-density regions of diffusion models, proposing a theoretical model for tracking processes and a high-density sampler, demonstrating that images generated by high-density sampling are often cartoonish or blurry and yield higher likelihood values than those obtained through ordinary sampling.
π― What it does: Proposes the Diffusion State-Guided Projected Gradient (DiffStateGrad) method, which enhances data consistency sampling and reduces artifacts in inverse problems by projecting the measurement gradient onto the low-rank subspace of the current diffusion state.
Diffusion-based Decoupled Deterministic and Uncertain Framework for Probabilistic Multivariate Time Series Forecasting
Qi Li (Beijing University of Posts and Telecommunications), Yong Zhang (Beijing University of Posts and Telecommunications)
CodeDiffusion modelTime Series
π― What it does: A decoupled deterministic and uncertainty framework (D U 3) based on diffusion models is proposed, which extracts high-confidence components through a pre-trained point prediction model, while the remaining high-uncertainty components are modeled for probability distribution using conditional DDPM, achieving dual capabilities of point prediction and probability prediction for long-term multivariate time series forecasting.
π― What it does: By combining a pre-trained video diffusion model and a multi-view diffusion model, dense multi-view multi-frame image matrices can be directly sampled without the need for additional training, thus achieving the generation of dynamic 3D content.
Digi-Q: Learning VLM Q-Value Functions for Training Device-Control Agents
Hao Bai (University of Illinois Urbana-Champaign), Aviral Kumar (Carnegie Mellon University)
CodeRobotic IntelligenceTransformerSupervised Fine-TuningReinforcement LearningVision Language ModelTabular
π― What it does: The Digi-Q method is proposed, which utilizes intermediate features generated by VLM to train the action value function Q through TD learning on offline data, and implements policy extraction through Best-of-N re-ranking to build an efficient control agent for mobile devices.
π― What it does: A robust fine-tuning method based on gradient direction projection, DiGraP, is proposed, balancing the robustness of pre-trained models with the performance of downstream tasks.