Conference on Neural Information Processing Systems · 1874 papers
Generalized Protein Pocket Generation with Prior-Informed Flow Matching
ZAIXI ZHANG, Qi Liu (University of Science and Technology of China)
CodeGenerationDrug DiscoveryProtein Structure PredictionTransformerFlow-based ModelMultimodalityBiomedical Data
🎯 What it does: A protein-ligand pocket generation model called PocketFlow is proposed and implemented, which can generate structurally reasonable and high-affinity protein pockets under the conditions of a given protein framework and different types of ligands (small molecules, peptides, RNA).
🎯 What it does: A hybrid model called WeatherGFT, which integrates physical equations and neural networks, has been developed to achieve finer-grained (30-minute) forecasts on hourly data.
Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search
Nicola Dainese (Aalto University), Pekka Marttinen (Aalto University)
CodeAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes the use of large language models (LLM) to generate world models in the form of Python code (Code World Models, CWM), and implements self-debugging and iterative generation through the GIF-MCTS method;
Generating Highly Designable Proteins with Geometric Algebra Flow Matching
Simon Wagner (Heidelberg Institute for Theoretical Studies), Jan Stuehmer
CodeProtein Structure PredictionFlow-based ModelBiomedical Data
🎯 What it does: A protein backbone generation model based on geometric algebra, GAFL, is proposed, which combines the Flow Matching framework and improves AlphaFold2's Invariant Point Attention (IPA) to Clifford Frame Attention (CFA), achieving more geometrically expressive message passing.
🎯 What it does: The GeNSIT framework is proposed, which utilizes neural networks and the Harris-Winston SDE to simultaneously learn spatial interaction models at both continuous and discrete levels, directly sampling and recovering the real OD matrix in the discrete OD matrix space.
🎯 What it does: A generative adversarial optimization framework using adaptive source critic regularization (aSCR) is proposed for offline model optimization, preventing out-of-domain predictions caused by model overfitting in the optimization trajectory.
🎯 What it does: A continuous-time fractional generative model (GFDM) driven by Markovian approximated fractional Brownian motion is proposed, and its reverse-time process is derived.
🎯 What it does: MDGEN is proposed, a molecular dynamics trajectory simulation framework based on generative models, supporting forward simulation, interpolation, upsampling, and filling incomplete trajectories;
Generative Modelling of Structurally Constrained Graphs
Manuel Madeira (École Polytechnique Fédérale de Lausanne), Pascal Frossard (École Polytechnique Fédérale de Lausanne)
CodeGenerationData SynthesisGraph Neural NetworkDiffusion modelGraphBiomedical Data
🎯 What it does: Proposes the ConStruct framework, which utilizes a graph discrete diffusion model to enforce hard structural constraints (such as planar graphs, acyclic graphs, etc.) during the generation process while maintaining consistency between training and sampling distributions.
Hezhe Qiao (Singapore Management University), Guansong Pang (Singapore Management University)
CodeAnomaly DetectionGraph Neural NetworkGraphFinance Related
🎯 What it does: In the context of semi-supervised graph anomaly detection, a learning framework called GGAD is proposed, which is based on generating pseudo-anomalous nodes to enhance anomaly detection performance when only a portion of normal nodes is provided.
🎯 What it does: A molecular optimization method combining domain-specific genetic algorithms with GFlowNet (Genetic GFN) is proposed, which generates high-reward molecules through genetic search and conducts unbiased sampling training using GFlowNet to enhance sample efficiency.
🎯 What it does: This paper presents GenRec, a unified video diffusion model capable of achieving high-quality video generation and video recognition within the same network, enhancing robustness in frame-missing scenarios through random frame conditioning and masking mechanisms.
🎯 What it does: A new framework is proposed that splits indoor panoramic semantic segmentation into oversampled areas (floor/ceiling) and undersampled areas, and jointly optimizes them using geometric information.
CodePose EstimationAutonomous DrivingOptimizationNeural Radiance FieldSimultaneous Localization and MappingPoint Cloud
🎯 What it does: Proposes the GeoNLF framework, which combines multi-view point cloud registration and pose-free LiDAR-NeRF for 3D scene reconstruction and viewpoint synthesis.
Get Rid of Isolation: A Continuous Multi-task Spatio-Temporal Learning Framework
Zhongchao Yi (University of Science and Technology of China), Yang Wang (University of Science and Technology of China)
CodeTransformerAuto EncoderTime Series
🎯 What it does: The CMuST (Continuous Multi-task Spatio-Temporal) framework is proposed, which enables joint learning of multiple spatio-temporal prediction tasks within the same urban system and achieves rapid adaptation to new tasks through continuous rolling training.
Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
Jiaxiang Li (University of Minnesota), Mingyi Hong (University of Minnesota)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: A reward learning and policy fine-tuning framework based on inverse reinforcement learning is proposed, which constructs a reward model using demonstration data and improves LLM alignment during the SFT phase.
GFT: Graph Foundation Model with Transferable Tree Vocabulary
Zehong Wang (University of Notre Dame), Yanfang Ye (University of Notre Dame)
CodeClassificationDomain AdaptationDrug DiscoveryGraph Neural NetworkContrastive LearningGraphBiomedical Data
🎯 What it does: This paper proposes a cross-task and cross-domain graph-based model GFT, utilizing computational trees (subtrees obtained from the message passing process) as transferable vocabulary to construct a pre-training and fine-tuning process.
GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning
Yanbin Wei (Southern University of Science and Technology), Yu Zhang (Hong Kong University of Science and Technology)
CodeGraph Neural NetworkLarge Language ModelVision Language ModelMultimodalityGraph
🎯 What it does: An end-to-end GITA framework is proposed, which renders graph structures into visual graphs and combines them with textual descriptions for graph reasoning.
Gliding over the Pareto Front with Uniform Designs
Xiaoyuan Zhang (City University of Hong Kong), Qingfu Zhang (City University of Hong Kong)
CodeOptimizationTabular
🎯 What it does: A unified multi-objective optimization method UMOD based on maximum packing design is proposed, which directly maximizes the minimum dual distance to achieve uniform coverage of the Pareto front.
🎯 What it does: A differentiable linear constraint satisfaction layer, GLinSAT, is proposed to project the output of neural networks into a feasible domain that satisfies general linear and bounded constraints.
🎯 What it does: The RMAB-G model is proposed to address the issue of traditional RMABs being unable to handle non-separable global rewards, and introduces linear and Shapley Whittle indices, as well as iterative and MCTS adaptive strategies;
Globally Q-linear Gauss-Newton Method for Overparameterized Non-convex Matrix Sensing
Xixi Jia (Xidian University), Defeng Sun (Hong Kong Polytechnic University)
CodeOptimization
🎯 What it does: An Approximate Gauss-Newton (AGN) method is proposed to solve the over-parameterized non-convex low-rank matrix sensing problem, and it is proven to achieve global Q-linear convergence from random initialization.
GO4Align: Group Optimization for Multi-Task Alignment
Jiayi Shen (University of Amsterdam), Marcel Worring (University of Amsterdam)
CodeOptimizationImage
🎯 What it does: This paper proposes GO4Align, a loss-oriented optimization method for aligning the learning progress of various tasks in multi-task learning through dynamic group allocation guided by task grouping and risk, aimed at alleviating the task imbalance problem.
Going Beyond Heuristics by Imposing Policy Improvement as a Constraint
Chi-Chang Lee (National Taiwan University), Pulkit Agrawal (Improbable AI Lab)
CodeRobotic IntelligenceReinforcement Learning
🎯 What it does: A new reinforcement learning method is proposed, which surpasses traditional heuristic reward methods by treating policy improvement as a constraint to enhance task performance in limited data situations.
GOMAA-Geo: GOal Modality Agnostic Active Geo-localization
Anindya Sarkar (Washington University in St. Louis), Yevgeniy Vorobeychik (Washington University in St. Louis)
CodeObject DetectionTransformerLarge Language ModelReinforcement LearningContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes a cross-modal, zero-shot active geographic localization (AGL) agent called GOMAA-Geo, which can achieve target localization using only aerial images for training, leveraging textual or ground image descriptions.
🎯 What it does: A simple and efficient baseline GoMatching is constructed, using the frozen image text detector DeepSolo for text detection and recognition, and tracking is performed through a lightweight rescoring head and a long-short term matching module LST-Matcher.
Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes
Xiaomeng Hu (Chinese University of Hong Kong), Tsung-Yi Ho (Chinese University of Hong Kong)
CodeAnomaly DetectionTransformerLarge Language ModelText
🎯 What it does: A two-stage 'Gradient Cuff' detection method based on the rejection loss function and its gradient features is proposed, which significantly improves the detection rate of various jailbreak attacks by large language models while maintaining a low false positive rate.
🎯 What it does: This paper proposes a gradient-guided optimization framework for diffusion models, utilizing a pre-trained diffusion model and the gradient of the objective function to generate samples that meet the objectives while preserving the latent low-dimensional structure of the data.
🎯 What it does: To address the model editing problem in graph neural networks, we propose Gradient Rewiring (GRE) and its improved version GRE+, which achieve rapid correction of prediction errors for single or consecutive nodes by making local adjustments using only the gradients of the target nodes, while ensuring that the training error does not increase.
Gradient-based Discrete Sampling with Automatic Cyclical Scheduling
Patrick Pynadath (Purdue University), Ruqi Zhang (Purdue University)
CodeOptimizationTransformerLarge Language ModelTextMultimodality
🎯 What it does: This paper proposes an Automatic Cyclical Sampler (ACS) for automatic periodic scheduling of gradient discrete sampling, which achieves efficient exploration and accurate sampling of multimodal discrete distributions by periodically adjusting the step size and balance parameters.
🎯 What it does: Achieved differentiable Lanczos and Arnoldi iterations for matrix functions (such as logarithmic determinants, matrix exponentials, etc.), providing closed-form adjoint systems;
Kanghee Park (University of Wisconsin-Madison), Loris D'Antoni (University of Wisconsin-Madison)
CodeGenerationTransformerLarge Language ModelText
🎯 What it does: This paper studies how to maintain the original probability distribution when generating text constrained by context-free grammar in large language models, proposing the Grammar-Aligned Decoding (GAD) problem;
GRANOLA: Adaptive Normalization for Graph Neural Networks
Moshe Eliasof (University of Cambridge), Haggai Maron (Technion and NVIDIA Research)
CodeGraph Neural NetworkGraph
🎯 What it does: A new graph neural network normalization layer called GRANOLA is proposed, which can adaptively normalize node features based on the graph structure.
Graph Classification via Reference Distribution Learning: Theory and Practice
Zixiao Wang (Chinese University of Hong Kong), Jicong Fan (Chinese University of Hong Kong)
CodeClassificationGraph Neural NetworkGraph
🎯 What it does: A new graph classification framework GRDL is proposed, which achieves efficient and accurate graph classification by directly comparing the distribution of node embeddings without using global pooling.
Graph Convolutions Enrich the Self-Attention in Transformers!
Jeongwhan Choi (Yonsei University), Noseong Park (KAIST)
CodeTransformerImageTextAudio
🎯 What it does: Redesign the self-attention of the Transformer as a graph filter and propose the GFSA mechanism to replace the attention layers in various Transformer models.
🎯 What it does: This paper proposes Graph Diffusion Policy Optimization (GDPO), a method that treats the reverse sampling of the Graph Diffusion Probabilistic Model (Graph DPM) as a Markov Decision Process and optimizes arbitrary (especially non-differentiable) reward objectives through reinforcement learning.
Graph Edit Distance with General Costs Using Neural Set Divergence
Eeshaan Jain (École Polytechnique Fédérale de Lausanne), Abir De (Indian Institute of Technology Bombay)
CodeGraph Neural NetworkGraph
🎯 What it does: A neural network framework is proposed that can approximate the graph edit distance (GED) while considering the general costs of any four types of edit operations.
🎯 What it does: This paper proposes a graph learning framework for numerical planning tasks, achieving efficient planning solutions by constructing interpretable graph features and learning heuristic functions.
🎯 What it does: This paper proposes Graph-tuple Neural Networks (GtNN), a multimodal graph neural network model that jointly processes multiple graphs sharing a common set of vertices.
Graph Neural Networks Need Cluster-Normalize-Activate Modules
Arseny Skryagin (Technical University of Darmstadt), Kristian Kersting (Technical University of Darmstadt)
CodeGraph Neural NetworkGraph
🎯 What it does: A pluggable Cluster-Normalize-Activate (CNA) module is proposed to enhance the expressive power of Graph Neural Networks (GNNs) and suppress the over-smoothing phenomenon.
CodeGenerationGraph Neural NetworkTransformerLarge Language ModelText
🎯 What it does: This paper proposes the Graph Uncertainty framework, which implements fine-grained uncertainty assessment of each claim in long text generation using semantic entailment graphs and graph centrality measures. This assessment is used to filter low-confidence claims during decoding, enhancing the factuality and informativeness of the text.
🎯 What it does: A graph autoencoder called GraphCroc based on cross-correlation decoding is proposed for structural reconstruction and downstream tasks in multi-graph scenarios.
CodeDomain AdaptationGraph Neural NetworkMixture of ExpertsGraph
🎯 What it does: A Mixture-of-Experts based graph neural network, GraphMETRO, is proposed to address complex and unknown graph distribution shifts in the real world.
GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration
Jiachen T. Wang (Princeton University), Ruoxi Jia (Virginia Tech)
CodeOptimizationComputational EfficiencyData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes an online batch selection algorithm called GREATS, which utilizes the Taylor expansion of gradients and greedy optimization to dynamically select the data that can most improve validation set performance during training, thereby accelerating the convergence of LLM training and enhancing generalization.
Grokking of Implicit Reasoning in Transformers: A Mechanistic Journey to the Edge of Generalization
Boshi Wang (Ohio State University), Huan Sun (Ohio State University)
CodeTransformerLarge Language Model
🎯 What it does: This study investigates the implicit reasoning ability of transformers on parameterized knowledge without explicit reasoning steps, observing the grokking phenomenon and systematic differences through synthetic combination and comparison tasks, revealing the formation of general reasoning circuits using logit lens and causal tracing.
🎯 What it does: This paper proposes a new class of structured matrices—GS matrices—and constructs structured orthogonal parameterization based on this matrix. Its effectiveness is then validated in various tasks (text understanding, text-to-image generation, 1-Lipschitz networks).
Group Robust Preference Optimization in Reward-free RLHF
Shyam Sundhar Ramesh (University College London), Ilija Bogunovic (University College London)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes the Group Robust Preference Optimization (GRPO) method, which aims to achieve fair and robust LLM fine-tuning for different user groups within a reward-agnostic RLHF framework.
🎯 What it does: A generative trajectory augmentation method (GTA) based on conditional diffusion models is proposed, which generates high-reward and dynamically feasible new trajectories to augment offline datasets by partially adding noise to the original trajectories and denoising under the guidance of amplified rewards.
GuardT2I: Defending Text-to-Image Models from Adversarial Prompts
Yijun Yang (Chinese University of Hong Kong), Qiang Xu (Chinese University of Hong Kong)
CodeGenerationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringImageText
🎯 What it does: A generative text censorship framework GUARDT2I has been designed and implemented, which converts text-guided embeddings into natural language through a conditional LLM, thereby detecting and preventing NSFW image generation caused by adversarial prompts.
Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization
Taeyoung Yun (Korea Advanced Institute of Science and Technology), Jinkyoo Park (Korea Advanced Institute of Science and Technology)
CodeOptimizationDiffusion modelTabularBenchmark
🎯 What it does: This paper proposes using conditional diffusion models to generate trajectories leading to high partitions in offline model optimization. It first constructs diverse improved trajectories through local search, then trains the diffusion model and incorporates contextual conditions and classifier-free guidance during sampling, and finally filters candidate designs using a proxy function.
🎯 What it does: An autoguidance method is proposed, which uses a low-quality version of the model itself to guide the high-quality model in the diffusion model, thereby improving image quality while maintaining diversity.
Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame
Evan Markou (Australian National University), Stephen Gould (Australian National University)
CodeClassificationOptimizationImage
🎯 What it does: This paper proposes a strategy for dynamically finding the nearest simplex equiangular tight frame (ETF) during the training process and using it as classifier weights, utilizing Riemannian optimization and deep declarative nodes for end-to-end training;
HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection
Xuefeng Du (University of Wisconsin), Yixuan Li (University of Wisconsin)
CodeClassificationTransformerLarge Language ModelText
🎯 What it does: Proposes the HaloScope framework, which utilizes unlabeled LLM-generated text that naturally occurs in the real world to estimate the authenticity of samples through subspace decomposition of internal activations, and trains a binary classifier for authenticity detection based on this.
Hamiltonian Monte Carlo Inference of Marginalized Linear Mixed-Effects Models
Jinlin Lai (University of Massachusetts), Daniel Sheldon (University of Massachusetts)
CodeComputational EfficiencyTabular
🎯 What it does: This paper proposes an efficient random effects marginalization algorithm for linear mixed effects models (LMM), significantly improving the sampling efficiency of Hamiltonian Monte Carlo (HMC).
🎯 What it does: This paper proposes a continuous general category discovery (C-GCD) task without sample replay, long-term multi-stage, and unknown old-new category ratios, and provides a complete learning framework.
Hardness of Learning Neural Networks under the Manifold Hypothesis
Bobak Kiani, Melanie Weber (Harvard University)
CodeClassificationRecognitionImage
🎯 What it does: This paper studies the feasibility of neural network learning under the manifold hypothesis and provides geometric conditions for learnable and unlearnable manifolds.
Harmonizing Visual Text Comprehension and Generation
Zhen Zhao (East China Normal University), Yuan Xie (East China Normal University)
CodeGenerationTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Developed the TextHarmony unified multimodal generation model, which balances visual text understanding and generation, and introduces Slide-LoRA to achieve modality consistency.
🎯 What it does: This study investigates the efficiency and sample efficiency of self-supervised visual pre-training, proposing a theoretical foundation and practical recommendations for low-dimensional projection heads and multi-view augmentation.
HAWK: Learning to Understand Open-World Video Anomalies
Jiaqi Tang (Hong Kong University of Science and Technology), Ying-Cong Chen (Hong Kong University of Science and Technology)
CodeAnomaly DetectionTransformerVision Language ModelOptical FlowVideoTextMultimodality
🎯 What it does: A video-language model named HAWK is proposed, specifically designed to understand and describe abnormal behaviors in videos, and supports interactive question answering.
🎯 What it does: This paper proposes a Hierarchical Clustering Graph Autoencoder (HC-GAE), which decomposes the graph into subgraphs through hard node assignment and compresses them using local convolution, then reconstructs them through soft node assignment to obtain representations that can be used for both node classification and graph classification.
HEALNet: Multimodal Fusion for Heterogeneous Biomedical Data
Konstantin Hemker (University of Cambridge), Mateja Jamnik (University of Cambridge)
CodeClassificationExplainability and InterpretabilityTransformerMultimodalityBiomedical Data
🎯 What it does: This paper proposes and implements HEALNet, a hybrid early fusion attention network that integrates multimodal (image, tabular, graph) data for survival analysis of multiple cancers.
HEPrune: Fast Private Training of Deep Neural Networks With Encrypted Data Pruning
Yancheng Zhang (University of Central Florida), Qian Lou (University of Central Florida)
CodeFederated LearningSafty and PrivacyComputational EfficiencyConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper presents HEPrune, a framework for implementing data pruning in a Fully Homomorphic Encryption (FHE) environment, making encrypted data training more efficient.
🎯 What it does: This paper proposes a heterogeneous graph label distribution learning (HGDL) framework that can predict the probability distribution of target nodes across multiple categories, achieving a finer-grained characterization of node functions compared to single-label or multi-label classification.
HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation
Bocheng, Yuhui Yin (360 AI Research)
CodeGenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper presents HiCo—a diffusion model based on a multi-branch, hierarchical control approach for generating high-quality images from layouts and text.
🎯 What it does: The HiCoM framework is proposed to achieve online reconstruction of multi-view streaming dynamic scenes, significantly improving training speed, rendering performance, and storage compression.
🎯 What it does: A multi-time scale gradient correction (MTGC) algorithm is designed for model drift correction in multi-level federated learning, addressing client and group-level model drift issues caused by multi-layer non-IID.
🎯 What it does: This paper proposes a new distance metric called Hierarchical Hybrid Sliced Wasserstein (H2SW), specifically designed for comparing heterogeneous joint distributions supported on different spaces, and experimentally validates its superiority in 3D mesh deformation, 3D mesh autoencoder training, and cross-dataset comparisons.
High Rank Path Development: an approach to learning the filtration of stochastic processes
Jiajie Tao (University College London), Chong Liu (ShanghaiTech University)
CodeGenerationData SynthesisGenerative Adversarial NetworkTime SeriesSequentialFinance Related
🎯 What it does: This paper proposes a high-order path development characteristic function (HRPCF) and defines a corresponding distance (HRPCFD) to measure extended weak convergence. It constructs an HRPCF-GAN for conditional time series generation and subsequently validates its effectiveness through hypothesis testing and generative model experiments.
High-Resolution Image Harmonization with Adaptive-Interval Color Transformation
Quanling Meng (Harbin Institute of Technology), Liqiang Nie (Harbin Institute of Technology)
CodeImage HarmonizationTransformerImage
🎯 What it does: This paper studies the harmonization problem of high-resolution images and proposes the Adaptive-Interval Color Transformation (AICT) method, which achieves pixel-level color transformation and adaptively adjusts the sampling interval through low-resolution predicted position-dependent 3D LUT.
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
Bernal Jimenez Gutierrez, Yu Su (Ohio State University)
CodeRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: A retrieval framework called HippoRAG is proposed based on the hippocampal memory index theory, which constructs a knowledge graph using LLM and achieves one-time multi-hop retrieval through Personalized PageRank, addressing the limitations of traditional RAG in knowledge integration.
HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction
Qianyue Hao (Tsinghua University), Yong Li (Tsinghua University)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes a scientific literature citation prediction work based on a hybrid language model, defines the core citation concept, and implements a two-stage retrieval + LLM reasoning pipeline.
Homology Consistency Constrained Efficient Tuning for Vision-Language Models
Huatian Zhang (University of Science and Technology of China), Zhendong Mao (University of Science and Technology of China)
CodeClassificationDomain AdaptationComputational EfficiencyTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a homotopy consistency constraint based on persistent homology for efficiently transferring large visual-language models in low-data environments, maintaining the structural consistency of the latent manifolds of images and text.
HonestLLM: Toward an Honest and Helpful Large Language Model
Chujie Gao (Mohamed Bin Zayed University of Artificial Intelligence), Xiangliang Zhang (University of Notre Dame)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: A framework is proposed to enhance the usefulness of large language models (LLMs) while maintaining honesty. It first defines the principle of honesty and constructs a specialized evaluation dataset called HONESET, and then designs two methods—training-free (curiosity-driven prompts) and two-stage fine-tuning—to improve the model's honesty and helpfulness.
🎯 What it does: In the task of visual text generation and editing, a multi-stage framework called TextGen based on ControlNet is proposed, which enhances the feature extraction of control information using Fourier-enhanced convolution and optimizes the output of control features through a frequency balancing mechanism, ultimately achieving a unified approach for multi-language text generation and editing.
How Do Large Language Models Acquire Factual Knowledge During Pretraining?
Hoyeon Chang (Korea Advanced Institute of Science and Technology), Minjoon Seo (Korea Advanced Institute of Science and Technology)
CodeTransformerLarge Language ModelText
🎯 What it does: This study investigates how large language models acquire factual knowledge during the pre-training phase, constructs a FICTIONAL KNOWLEDGE dataset, and measures the processes of memory acquisition and forgetting through the insertion of training instances.
How do Large Language Models Handle Multilingualism?
Yiran Zhao (National University of Singapore), Lidong Bing (DAMO Academy, Alibaba Group)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the processing mechanisms of large language models in multilingual tasks, proposes a three-stage multilingual workflow (MWork), and designs an unsupervised method for detecting parallel language-specific neurons (PLND) to validate and utilize these neurons for multilingual enhancement with limited data.
How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad
Emmanuel Abbe (Apple), Omid Saremi (Apple)
CodeTransformerSequential
🎯 What it does: This paper explores the 'global reasoning barrier' of Transformers during zero-shot learning from both theoretical and experimental perspectives. It introduces a globality metric to measure the globality of data distribution and proves that tasks with high globality are difficult for Transformers to learn effectively. Subsequently, it introduces the scratchpad technique, particularly the inductive scratchpad, demonstrating its significant effect on improving OOV/length generalization.
How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective
Qiaozhe Zhang (Huazhong University of Science and Technology), Yingzhuang Liu (Huazhong University of Science and Technology)
CodeOptimizationConvolutional Neural NetworkImage
🎯 What it does: The theoretical limits of deep network pruning were studied, providing lower and upper bounds for the pruning ratio, which were experimentally verified to be nearly consistent;
🎯 What it does: A text-to-image diffusion model (CIDM) capable of continuous learning of multiple custom concepts is proposed, addressing the challenges of catastrophic forgetting and concept neglect in the context of Concept Incremental Customization (CIFC).
How to Solve Contextual Goal-Oriented Problems with Offline Datasets?
Ying Fan (University of Wisconsin-Madison), Ching-An Cheng (Microsoft Research)
CodeOptimizationReinforcement LearningSequential
🎯 What it does: This paper proposes a Contextual goal-Oriented Data Augmentation (CODA) method to address the contextual goal-oriented problem in offline data using known trajectories and context-goal pairs.
🎯 What it does: This paper proposes a correction of the score distillation sampling (SDS) of diffusion models using a sparse perspective linear prior (inline prior) to achieve new view synthesis under a sparse perspective, and combines this method with 3D Gaussian Splatting (3DGS);
Rohan Alur (Massachusetts Institute of Technology), Devavrat Shah (Massachusetts Institute of Technology)
CodeClassificationRecognitionConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: A framework based on algorithm indistinguishable subsets is proposed, utilizing expert feedback to improve predictions, and a method is provided to test whether expert information can be captured by the algorithm.
🎯 What it does: A visual and language-based physical humanoid robot object rearrangement system, HumanVLA, has been developed, utilizing a teacher-student framework to distill the teacher policy obtained from reinforcement learning into a student model that can execute tasks solely based on first-person vision and natural language instructions.
CodeRecognitionGenerationData SynthesisSafty and PrivacyTransformerLarge Language ModelGenerative Adversarial NetworkContrastive LearningText
🎯 What it does: A human-readable fingerprint (HuRef) is proposed for large language models (LLM) by extracting invariants from model parameters and mapping them to natural images, publicly disclosing the fingerprint without leaking parameters, and ensuring the authenticity of the fingerprint through zero-knowledge proofs.
🎯 What it does: GEMCODE is proposed and implemented, a collaborative generation pipeline that integrates generative models and evolutionary optimization for the design of co-crystal ligands with target formulation properties (such as suppressibility) based on drug molecules;
Hybrid Top-Down Global Causal Discovery with Local Search for Linear and Nonlinear Additive Noise Models
Sujai Hiremath (Cornell University), Kyra Gan (Cornell University)
CodeTabular
🎯 What it does: This paper proposes a hybrid causal discovery framework that combines functional causal models with constraint search. It provides hierarchical topological sorting algorithms (LHTS, NHTS) for linear and nonlinear additive noise models, as well as a non-parametric edge pruning algorithm (ED) based on local conditional sets, achieving precise reconstruction of global causal graphs.
🎯 What it does: In federated learning, this paper investigates and addresses the issue of knowledge distillation methods amplifying the impact of model poisoning attacks, and proposes the HYDRA-FL hybrid distillation technique, which mitigates attack amplification by utilizing shallow distillation and reducing the KD weight of the final layer, while maintaining or improving performance in non-attack scenarios.
Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers
Sukjun Hwang (Carnegie Mellon University), Albert Gu (Carnegie Mellon University)
CodeTransformerImageText
🎯 What it does: A framework that unifies sequence mixers into matrix mixers is proposed, and based on this, a new bidirectional state space model called Hydra is designed as a scalable, data-dependent, and efficient sequence mixer.
HYDRA: Model Factorization Framework for Black-Box LLM Personalization
Yuchen Zhuang (Georgia Institute of Technology), Bo Dai (Georgia Institute of Technology)
CodeGenerationRetrievalRecommendation SystemTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: The HYDRA framework is proposed, which combines retrieval-re-ranking with a black-box LLM adapter to achieve personalized generation of user behavior history without accessing model parameters.
Janek Haberer (Kiel University), Olaf Landsiedel (Kiel University)
CodeClassificationRecognitionComputational EfficiencyTransformerMixture of ExpertsImage
🎯 What it does: A scalable Vision Transformer (HydraViT) is proposed, which generates multiple sub-networks by randomly cropping attention heads and embedding dimensions during training.
🎯 What it does: This paper proposes a complete scheme for embedding supervised models into hyperbolic geometry, primarily focusing on decision trees and their ensemble models.
HyperLogic: Enhancing Diversity and Accuracy in Rule Learning with HyperNets
Yang Yang (Chinese University of Hong Kong), Shuang Li (Chinese University of Hong Kong)
CodeClassificationExplainability and InterpretabilityMixture of ExpertsTabularFinance Related
🎯 What it does: Proposes the HyperLogic framework, which uses hypernetworks to generate weights for the main rule learning network, thereby enhancing the diversity and accuracy of rule learning while maintaining interpretability.
I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing
Yiwei Ma (Xiamen University), Rongrong Ji (Xiamen University)
CodeImage TranslationGenerationData SynthesisLarge Language ModelPrompt EngineeringImageTextBenchmark
🎯 What it does: I2EBench is proposed, which includes a comprehensive evaluation benchmark with over 2000 images and 4000 instructions for assessing instruction-based image editing models.
Identifiability Guarantees for Causal Disentanglement from Purely Observational Data
Ryan Welch (Massachusetts Institute of Technology), Caroline Uhler (Massachusetts Institute of Technology)
CodeOptimizationScore-based ModelGraph
🎯 What it does: This paper proposes a theory and algorithm for causal identifiability using the zero variance property of the score function under a model with only observational data, linear mixing, and nonlinear Gaussian noise.
🎯 What it does: This paper studies system identification of linear parameterized nonlinear systems with real analytic characteristic functions under non-active exploration conditions, providing the non-asymptotic convergence rates of LSE and SME, and verifying the convergence rates through pendulum and quadrotor simulation experiments consistent with the theory;