Conference on Empirical Methods in Natural Language Processing · 435 papers
Delving into Qualitative Implications of Synthetic Data for Hate Speech Detection
Camilla Casula (Fondazione Bruno Kessler), Sara Tonelli (Fondazione Bruno Kessler)
CodeClassificationData SynthesisTransformerLarge Language ModelText
🎯 What it does: Evaluated and qualitatively analyzed the effectiveness of synthetic data generated by large language models (LLMs) in English hate speech detection, comparing model performance using original data, pure synthetic data, and synthetic-original hybrid training. Manual annotations were conducted on the authenticity of synthetic text, preservation of hate labels, and distribution of identity targets.
DEM: Distribution Edited Model for Training with Mixed Data Distributions
Dhananjay Ram (AGI Foundations, Amazon AWS AI Labs), Sheng Zha (AGI Foundations, Amazon AWS AI Labs)
CodeComputational EfficiencyData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the Distribution Edited Model (DEM), which first fine-tunes each data source separately and extracts parameter differences between the data source and the base model (distribution vectors), then combines them with weighted sums to obtain a multi-task instruction model.
🎯 What it does: Map dialog sentences to a latent space of action clusters through a pre-trained sentence embedding model (Dialog2Flow), enabling automatic extraction of task-oriented dialogue workflows.
Distract Large Language Models for Automatic Jailbreak Attack
Zeguan Xiao (Shanghai University of Finance and Economics), Yun Chen (Shanghai University of Finance and Economics)
CodeAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a black-box automated 'Distraction based Adversarial Prompts (DAP)' framework to generate generalizable jailbreak templates that can induce LLMs to produce harmful outputs without explicitly containing specific malicious instructions.
DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions
Nigel Fernandez (University of Massachusetts Amherst), Andrew Lan (Eedi)
CodeData SynthesisRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningAuto EncoderText
🎯 What it does: Proposed DiVERT, which leverages a variational autoencoder framework to learn interpretable error text representations and generates corresponding mathematical MCQ distractors based on these errors.
DKEC: Domain Knowledge Enhanced Multi-Label Classification for Diagnosis Prediction
Xueren Ge (University of Virginia), Homa Alemzadeh (University of Virginia)
CodeClassificationRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelTextBiomedical DataElectronic Health RecordsChain-of-Thought
🎯 What it does: Propose a diagnostic prediction method based on automatically constructing a heterogeneous medical knowledge graph and employing a label-level attention mechanism in multi-label text classification.
Jialiang Xu (Stanford University), Denghui Zhang (Stevens Institute of Technology)
CodeSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Studied whether large language models (LLMs) comply with copyright regulations when user inputs contain copyright information, and constructed a benchmark dataset consisting of 43,200 simulated queries, conducting experiments across different content types, copyright notice types, and text lengths.
Do LLMs suffer from Multi-Party Hangover? A Diagnostic Approach to Addressee Recognition and Response Selection in Conversations
Nicolò Penzo (Fondazione Bruno Kessler), Marco Guerini (Fondazione Bruno Kessler)
CodeClassificationRecognitionSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringTextGraph
🎯 What it does: Propose a diagnostic method to evaluate the zero-shot performance of large language models (LLMs) in multi-party conversation (MPC) tasks for response selection (RS) and addressee recognition (AR), focusing on the contribution of text versus structural information and prompt sensitivity.
Do Text-to-Vis Benchmarks Test Real Use of Visualisations?
Hy Nguyen (University of Sydney), Jonathan K. Kummerfeld (University of Sydney)
CodeTextBenchmark
🎯 What it does: This paper analyzes public code and existing Text-to-Vis benchmarks to explore whether benchmark datasets truly reflect users' actual usage in four visualization libraries: Python, R, JavaScript, and Vega-Lite.
Do We Need Language-Specific Fact-Checking Models? The Case of Chinese
Caiqi Zhang (University of Cambridge), Andreas Vlachos (University of Cambridge)
CodeClassificationRetrievalAdversarial AttackTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Explores whether a dedicated fact-checking model for Chinese is necessary, proposes a Chinese document-level evidence retriever (DLR), and constructs an adversarial dataset based on CHEF to test the model's robustness against language and cultural biases.
Does Large Language Model Contain Task-Specific Neurons?
Ran Song (Kunming University of Science and Technology), Zhengtao Yu (Kunming University of Science and Technology)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Investigate and verify the existence of task-specific neurons in LLMs, and propose a method based on causal gradient variation with special tokens to locate them.
DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging
Tzu-Han Lin (National Taiwan University), Yun-Nung Chen (National Taiwan University)
CodeAI Code AssistantReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningText
🎯 What it does: Propose a method called DogeRM that integrates domain knowledge into a reward model by merging a pre-trained reward model with a domain-specific SFT language model through weighted averaging, thereby enhancing the domain performance of the reward model without requiring additional preference data.
Don’t Forget Your Reward Values: Language Model Alignment via Value-based Calibration
Xin Mao (Nanyang Technological University), Anh Tuan Luu (SEA Group)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningContrastive LearningText
🎯 What it does: Propose a reward-based calibration method called VCB for directly aligning large language models, avoiding the limitations of traditional sequence-based calibration methods.
Dynamic Multi-granularity Attribution Network for Aspect-based Sentiment Analysis
Yanjiang Chen (University of Science and Technology of China), Qi Liu (University of Science and Technology of China)
CodeClassificationExplainability and InterpretabilityGraph Neural NetworkTransformerLarge Language ModelText
🎯 What it does: Proposed a Dynamic Multi-Granularity Attribution Network (DMAN), which leverages Integrated Gradients to extract multi-step attribution information, integrates token-level and span-level attributions, and dynamically focuses on syntactic graphs to enhance Aspect-based Sentiment Analysis (ABSA) performance.
Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models
Somanshu Singla (University Of California San Diego), Eric P. Xing (Mohamed Bin Zayed University Of Artificial Intelligence)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerPrompt EngineeringText
🎯 What it does: Proposed the Dynamic Reward and Prompt Optimization (DRPO) method, achieving self-alignment without additional training, significantly enhancing the performance of large language models (LLMs) on alignment tasks.
🎯 What it does: Proposes the DYNAMICER benchmark to evaluate the identification of emerging entity mentions in a continuously updated knowledge base and their alignment to dynamic entities; and designs a Temporal Segmented Clustering with Continuous Adaptation (TempCCA) method based on time-segmented clustering and continuous adaptation to address novel mentions;
🎯 What it does: Proposed and implemented the DynaThink framework, enabling large language models to dynamically select 'fast thinking' or 'slow thinking' modes during reasoning tasks, thereby optimizing resource consumption while maintaining accuracy.
DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities
Thong Nguyen (University of Amsterdam), Andrew Yates (University of Amsterdam)
CodeRetrievalTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes the DyVo model, which integrates Wikipedia entity embeddings into the vocabulary by dynamically incorporating entity candidates through a dynamic vocabulary head within the Learned Sparse Retrieval (LSR) framework, enabling both queries and document representations to simultaneously include word fragments and entities.
ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness?
Siddhant Waghjale (Carnegie Mellon University), Daniel Fried (Carnegie Mellon University)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Proposes the ECCO benchmark to evaluate the ability of large language models to generate efficient Python code while maintaining functional correctness; simultaneously implements a reproducible cloud execution platform, JUDGE0, to stably measure runtime and memory consumption.
CodeGenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: This study proposes the task of generating entity-centric information retrieval questions (ECIS) from videos and constructs a new dataset called VIDEOQUESTIONS;
🎯 What it does: This paper proposes an entity disambiguation method based on adversarial counterfactual training (CFT), which uses counterfactual samples generated by masking the surface features of entities in the input to suppress the model's 'shortcut learning' reliance on surface characteristics, thereby improving disambiguation performance for shadowed entities.
🎯 What it does: Proposed an efficient soft prompt tuning method to adapt multilingual pre-trained language models (mPLMs) for zero-shot cross-lingual transfer tasks on unseen languages.
EFUF: Efficient Fine-Grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Models
Shangyu Xing (Nanjing University), Xinyu Dai (Nanjing University)
CodeLarge Language ModelVision Language ModelContrastive LearningMultimodality
🎯 What it does: Proposed an EFUF framework that utilizes unpaired data and gradient ascent to mitigate hallucinations in multi-modal large language models.
EH-MAM: Easy-to-Hard Masked Acoustic Modeling for Self-Supervised Speech Representation Learning
Ashish Seth (University of Maryland, College Park), Dinesh Manocha (University of Maryland, College Park)
CodeRepresentation LearningTransformerAudio
🎯 What it does: Propose EH-MAM, a self-supervised speech representation learning framework that automatically identifies audio frames difficult to reconstruct through a loss predictor in the teacher network, and gradually masks them during training to create more challenging pre-training tasks.
Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Properties
Keunwoo Peter Yu (University of Michigan), Joyce Chai (University of Michigan)
CodeData-Centric LearningTransformerLarge Language ModelVision Language ModelVideoText
🎯 What it does: Developed an incremental contextual learning training paradigm called EILeV for video and text, utilizing distributed attributes to achieve few-shot learning for video description.
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence
Junru Lu (University of Warwick), Xing Sun (Tencent YouTu Lab)
CodeOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper investigates the verbosity issue in Direct Preference Optimization (DPO) when generating text and proposes an improved method called SamPO. SamPO eliminates algorithmic bias toward response length by undersampling the KL divergence of DPO, thereby reducing the model's output verbosity and improving alignment.
Embedded Named Entity Recognition using Probing Classifiers
Nicholas Popovic, Michael Färber (TU Dresden)
CodeRecognitionTransformerLarge Language ModelText
🎯 What it does: Propose EMBER, which achieves zero-shot streaming named entity recognition by leveraging the internal representations of pre-trained decoder language models and a probe classifier;
Emotion Granularity from Text: An Aggregate-Level Indicator of Mental Health
Krishnapriya Vishnubhotla (University of Toronto), Saif M. Mohammad (National Research Council Canada)
CodeText
🎯 What it does: This paper constructs user emotion time series (emotion arcs) from social media texts such as tweets and Reddit posts, calculates the emotion granularity (EG) metric by computing the Spearman correlation between emotion pairs, and compares differences between various mental health condition (MHC) groups and the control group on this metric.
🎯 What it does: Proposed and implemented the EmphAssess evaluation framework for automatically assessing the performance of speech-to-speech (S2S) models in preserving and transferring prosody.
Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective
Hanqi Yan (King's College London), Yulan He (King's College London)
CodeExplainability and InterpretabilityRepresentation LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Explores the significance of monosemanticity in large language models and enhances model alignment performance by incorporating feature decorrelation regularization (DecPO) into DPO training.
Enhancing High-order Interaction Awareness in LLM-based Recommender Model
Xinfeng Wang (University of Yamanashi), Yoshimi Suzuki (University of Yamanashi)
CodeRecommendation SystemGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTextGraph
🎯 What it does: This paper proposes an enhanced LLM recommendation model, ELMRec, which improves recommendation performance by integrating graph structural information and re-ranking methods.
Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs
Cheng Gao (Tsinghua University), Maosong Sun (Tsinghua University)
CodeData SynthesisRetrievalLarge Language ModelText
🎯 What it does: This paper uses large language models to automatically extract key information from criminal verdict texts, generate short anonymized queries, and combines knowledge-driven data augmentation to construct the largest Chinese legal case retrieval dataset, LEAD.
Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration
Kangxi Wu (Key Laboratory of AI Safety, Chinese Academy of Sciences Institute of Computing Technology), Xueqi Cheng (Key Laboratory of AI Safety, Chinese Academy of Sciences Institute of Computing Technology)
CodeExplainability and InterpretabilityLarge Language ModelContrastive LearningTextBenchmark
🎯 What it does: For the training data attribution task in large-scale language models, the Debias and Denoise Attribution (DDA) method is proposed, improving traditional influence functions to reduce the impact of training errors on attribution results.
EPO: Hierarchical LLM Agents with Environment Preference Optimization
Qi Zhao (Brown University), George Konidaris (Brown University)
CodeOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AIVision Language ModelVision-Language-Action ModelImageTextMultimodalityBenchmark
🎯 What it does: Propose a hierarchical large language model framework that utilizes subgoal decomposition and low-level action generation to address long-horizon decision-making problems, and automatically generates reward signals through environmental feedback;
🎯 What it does: This paper evaluates the performance of multilingual large language models in English-Amharic bidirectional translation and conducts a systematic analysis of translation errors.
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the ESC-Eval framework, leveraging role-playing models and multi-turn dialogue generation to evaluate the performance of large language models in emotional support conversations (ESC), and train an automatic scoring model ESC-RANK based on human annotation.
Evaluating n-Gram Novelty of Language Models Using Rusty-DAWG
William Merrill (New York University), Yanai Elazar (Allen Institute for AI)
CodeGenerationComputational EfficiencyData-Centric LearningTransformerLarge Language ModelTextSequential
🎯 What it does: Built a search tool called RUSTY-DAWG based on CDAWG, which can perform arbitrary-length n-gram matching on large-scale pre-trained corpora (e.g., Pile) in constant time, and used this tool to evaluate the n-gram novelty of language model-generated text;
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper designs bias-free prompts and systematically evaluates and compares the psychological safety and well-being of five mainstream large language models (GPT-3, InstructGPT, GPT-3.5, GPT-4, Llama-2-chat-7B) using psychological assessments (Short Dark Triad, Big Five Inventory, Flourishing Scale, Satisfaction With Life Scale).
Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Models
Yi Zhou (Cardiff University), Jose Camacho-Collados (Cardiff University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextTime SeriesBenchmark
🎯 What it does: Studied the changes in social bias over time for masked language models trained on social media data, systematically evaluating temporal fluctuations in bias using the AULA metric on CrowS-Pairs and StereoSet.
Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering
Sungho Ko (Yonsei University), Dongha Lee (Yonsei University)
CodeKnowledge DistillationRepresentation LearningTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextGraphRetrieval-Augmented Generation
🎯 What it does: This study proposes an evidence-focused factual summarization framework named EFSUM to enhance zero-shot question answering systems based on knowledge graphs;
Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently
Kanishka Misra (University of Texas at Austin), Kyle Mahowald (University of Texas at Austin)
CodeTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Systematically evaluate the performance of language models on attribute inheritance reasoning tasks by incorporating experimental context (example prompts and instructions) into the COMPS dataset.
Exploring Intra and Inter-language Consistency in Embeddings with ICA
Rongzhi Li (University of Tokyo), Hitomi Yanaka (University of Tokyo)
CodeExplainability and InterpretabilityRepresentation LearningText
🎯 What it does: Investigated the reproducibility of independent components within monolingual word vectors and their cross-lingual correspondences, using Icasso to verify the stability of independent components within the same language and statistical tests to confirm the consistency of semantic axes across languages.
Exploring Intrinsic Language-specific Subspaces in Fine-tuning Multilingual Neural Machine Translation
Zhe Cao (Nara Institute of Science and Technology), Taro Watanabe (Nara Institute of Science and Technology)
CodeTransformerSupervised Fine-TuningText
🎯 What it does: Investigated the intrinsic low-rank subspaces of each language in multilingual neural machine translation models during fine-tuning, proposing language-specific LoRA (LSLo) to efficiently separate subspaces and combine gradient progressive pruning and architecture learning for parameter-efficient fine-tuning.
Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems
Jun Zhao (Fudan University), Xuanjing Huang (Fudan University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Investigated the compositional generalization ability of large language models (LLMs) in mathematical reasoning, proposed a novel trap-style problem set called MATHTRAP, and experimentally evaluated model performance in handling logical traps.
Exploring the Practicality of Generative Retrieval on Dynamic Corpora
Chaeeun Kim (KAIST AI), Minjoon Seo (KAIST AI)
CodeRetrievalTransformerTime SeriesBenchmark
🎯 What it does: Evaluate and compare the adaptability, robustness, and efficiency of generative retrieval and dual-encoder retrieval under dynamic corpora, proposing the DynamicIR framework.
Exploring Union and Intersection of Visual Regions for Generating Questions, Answers, and Distractors
Wenjian Ding (Nankai University), Zhenglu Yang (Nankai University)
CodeGenerationRecurrent Neural NetworkTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose a unified framework named ReBo for generating visual multiple-choice questions, answers, and distractors (QAD), leveraging a cyclic multimodal encoder and combining union and intersection scores of image regions to control the diversity and coverage of generated content.
Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction
Bowen Zhang (National University of Singapore), Harold Soh (National University of Singapore)
CodeTransformerLarge Language ModelPrompt EngineeringTextGraphRetrieval-Augmented Generation
🎯 What it does: Proposes a three-stage framework EDC (Extract-Define-Canonicalize), decomposing knowledge graph construction into open-information extraction, schema definition, and schema canonicalization, while introducing Schema Retriever for retrieval enhancement;
Fewer is More: Boosting Math Reasoning with Reinforced Context Pruning
Xijie Huang (Hong Kong University of Science and Technology), Mao Yang (Hong Kong University of Science and Technology)
CodeComputational EfficiencyTransformerLarge Language ModelReinforcement LearningTextBenchmarkChain-of-Thought
🎯 What it does: Designed and implemented the CoT-Influx plugin, utilizing a two-step pruning approach (from coarse to fine) to remove redundant CoT examples and tokens, maximizing input volume to enhance LLM mathematical reasoning performance.
Finding Blind Spots in Evaluator LLMs with Interpretable Checklists
Sumanth Doddapaneni (Nilekani Centre at AI4Bharat), Mitesh M Khapra
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes the FBI framework, which evaluates the capability of LLMs in text generation tasks by constructing targeted perturbation checks, focusing on factual accuracy, instruction following, long-text coherence, and reasoning ability.
Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary Debates
Aida Kostikova (Bielefeld University), Steffen Eger (University of Mannheim)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningText
🎯 What it does: This study constructs a fine-grained cohesion and anti-cohesion annotation dataset for German parliamentary debate texts from 1867 to 2022, leveraging large language models such as GPT-4 to achieve high-quality automated annotation, revealing the temporal evolution of cohesion types in discussions about women and immigration topics;
Fine-Grained Prediction of Reading Comprehension from Eye Movements
Omer Shubi (Technion Israel Institute of Technology), Yevgeni Berzak (Technion Israel Institute of Technology)
CodeClassificationConvolutional Neural NetworkRecurrent Neural NetworkTransformerLarge Language ModelTextMultimodalityTime Series
🎯 What it does: This study predicts the accuracy of single-paragraph single-question reading comprehension using eye movement data, proposing and systematically evaluating three Transformer-based text-eye movement fusion models.
Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models
Jeonghwan Kim (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
CodeRecognitionExplainability and InterpretabilityPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper analyzes the shortcomings of large vision-language models optimized with instruction tuning in fine-grained visual classification tasks, and proposes the attribute-based multi-grained benchmark FINER and the ATTRSEEK prompting scheme to enhance zero-shot recognition and interpretability performance.
Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models
Sander Land (Cohere), Max Bartolo (Cohere)
CodeAnomaly DetectionTransformerLarge Language ModelText
🎯 What it does: This paper proposes an automatic method to detect 'untrained/under-trained' tokens (glitch tokens) in large language models, along with an open-source tool.
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document
Joonho Yang (Chung-Ang University), Hwanhee Lee (Chung-Ang University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the FIZZ system, which detects factual consistency in abstract summaries by leveraging core citation parsing, LLM-generated atomic facts, NLI scoring, and adaptive granularity expansion.
Flee the Flaw: Annotating the Underlying Logic of Fallacious Arguments Through Templates and Slot-filling
Irfan Robbani (JAIST), Kentaro Inui (MBZUAI)
CodeClassificationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Developed a logic fallacy structure annotation scheme based on templates and slot filling, and constructed the FtF dataset containing 400 LOGIC arguments.
Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models
Xinyu Liu (Northeastern University), JingBo Zhu
CodeExplainability and InterpretabilityComputational EfficiencyRecurrent Neural NetworkTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed the Forgetting Curve method, evaluating the memory capacity of long-context language models by comparing replication tasks with language model accuracy curves, and conducting systematic experiments on multiple models.
Free your mouse! Command Large Language Models to Generate Code to Format Word Documents
Shihao Rao (Chinese Academy of Sciences), Can Ma (Chinese Academy of Sciences)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes an automatic Word document formatting method based on LLM called TEXT-TO-FORMAT, and constructs an evaluation dataset named DOCFORMEVAL.
FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs
Yiyuan Li (UNC-Chapel Hill), Pengfei Liu (Shanghai Jiao Tong University)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose and construct the FROG benchmark to evaluate the performance of large language models on fuzzy reasoning tasks involving generalized quantifiers.
GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities
Sreyan Ghosh (University of Maryland), Dinesh Manocha (University of Maryland)
CodeClassificationRecognitionGenerationTransformerLarge Language ModelPrompt EngineeringTextMultimodalityAudio
🎯 What it does: Proposed GAMA, a general large audio-language model integrating multiple audio features (Audio Q-Former, AST + multi-layer aggregator) and LLM, achieving complex reasoning capabilities through the synthesized CompA-R dataset;
Game on Tree: Visual Hallucination Mitigation via Coarse-to-Fine View Tree and Game Theory
Xianwei Zhuang (Peking University), Yuexian Zou (Peking University)
CodeExplainability and InterpretabilityComputational EfficiencyVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose a training-agnostic, plug-and-play decoding strategy called GTHM to alleviate visual hallucinations in large vision-language models.
Generate-on-Graph: Treat LLM as both Agent and KG for Incomplete Knowledge Graph Question Answering
Yao Xu (University of Chinese Academy of Sciences), Kang Liu (University of Chinese Academy of Sciences)
CodeGenerationTransformerLarge Language ModelAgentic AIPrompt EngineeringTextGraphRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes a new task called IKGQA, which uses large language models (LLMs) for question answering under incomplete knowledge graphs (IKGs), and constructs a corresponding dataset; simultaneously, it introduces the Generate-on-Graph (GoG) method, which employs LLMs as both an Agent and a KG, exploring and generating missing facts within the knowledge graph through a Thinking-Searching-Generating framework to address missing knowledge issues.
Yanting Liu (East China Normal University), Xiaoling Wang (East China Normal University)
CodeGenerationDomain AdaptationTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Propose a dynamically expandable vocabulary language model framework that uses a trainable phrase encoder to treat any text fragment as a single generation unit, enabling the insertion of dynamic phrases during generation.
Generative Subgraph Retrieval for Knowledge Graph–Grounded Dialog Generation
Jinyoung Park (Korea University), Hyunwoo J. Kim (Korea University)
CodeGenerationRetrievalTransformerLarge Language ModelGraphRetrieval-Augmented Generation
🎯 What it does: Designed a dialogue generation model called DialogGSR based on generative subgraph retrieval, which can directly generate knowledge graph subgraph sequences on language models and integrate them with dialogue history to generate responses.
CodeRetrievalTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Designed a zero-shot retrieval pipeline GENRA, which leverages LLM to generate multiple prompt sentences, first performs query expansion and retrieval, then uses LLM for relevance assessment, and finally conducts multi-document retrieval on each validated document and obtains the final ranking through rank aggregation.
Global Reward to Local Rewards: Multimodal-Guided Decomposition for Improving Dialogue Agents
Dong Won Lee (Massachusetts Institute of Technology), Louis-Philippe Morency (Carnegie Mellon University)
CodeKnowledge DistillationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningMultimodality
🎯 What it does: Propose a framework (GELI) that decomposes a single global explicit reward into local episode-level rewards, leveraging multimodal implicit feedback to guide the decomposition process and improve long-horizon dialogue agents.
Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMs
Chengyuan Liu (Zhejiang University), Fei Wu (Zhejiang University)
CodeDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied the use of vocabulary expansion in domain-specific LLMs and proposed an adaptive method called VEGAD to select the most valuable subset of vocabulary
GOME: Grounding-based Metaphor Binding With Conceptual Elaboration For Figurative Language Illustration
Linhao Zhang (Chinese Academy of Sciences), Guangluan Xu (Chongqing University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageTextChain-of-Thought
🎯 What it does: Proposes a system named GOME for visualizing language metaphors from a grounding perspective. It first generates refined visual interpretations using LLMs (triggered by chain-of-thought prompts to elicit rhetorical knowledge), then employs cross-attention constraints during the Stable Diffusion process to bind metaphor attributes to target objects, resulting in images more aligned with metaphorical meanings.
GRASS: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients
Aashiq Muhamed (Carnegie Mellon University), Virginia Smith (Carnegie Mellon University)
CodeOptimizationComputational EfficiencyLarge Language ModelTextBenchmark
🎯 What it does: Propose a subspace optimization method called GRASS that utilizes sparse projection to significantly reduce the memory consumption of optimizer states and gradients during full-parameter training of LLMs, while maintaining approximately the same training effectiveness.
GuardBench: A Large-Scale Benchmark for Guardrail Models
Elias Bassani (European Commission Joint Research Centre), Ignacio Sanchez (European Commission Joint Research Centre)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Propose GuardBench, a benchmark consisting of 40 security evaluation datasets, to systematically assess and compare guardrail models of generative AI.
Hate Personified: Investigating the role of LLMs in content moderation
Sarah Masud (IIIT Delhi), Tanmoy Chakraborty (Technische Universität München)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Investigate the sensitivity of LLMs to geographical location, identity attributes, and numerical anchoring contexts in content moderation, and explore their impact on the consistency of subjective annotations.
Sanne Hoeken (Bielefeld University), Özge Alacam (Bielefeld University)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the Hateful Word in Context Classification (HateWiC) task and construct a dataset with approximately 4,000 word-context instances to detect whether a word's meaning in a specific context carries hateful connotations.
Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization
Miyoung Ko (KAIST), Minjoon Seo (KAIST)
CodeExplainability and InterpretabilityGraph Neural NetworkLarge Language ModelPrompt EngineeringText
🎯 What it does: Built a hierarchical graph-based framework that decomposes complex real-world problems into subproblems at D1, D2, and D3 levels to analyze the reasoning process of large language models (LLMs).
HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy
YongKang Liu, Hinrich Schuetze
CodeOptimizationComputational EfficiencyLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a hierarchical full-parameter fine-tuning strategy named HiFT, which updates only part of the model's layers at each step, significantly reducing GPU memory usage and enabling full-parameter fine-tuning of 7B models on 24G GPUs.
Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction
Jinchuan Zhang (Chinese Academy of Sciences), Songlin Hu (Chinese Academy of Sciences)
CodeSafty and PrivacyAdversarial AttackReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AIPrompt EngineeringText
🎯 What it does: Propose the HARM framework to achieve top-down test case generation based on fine-grained risk classification and multi-round dialogue red team attacks, and enhance LLM safety through post-detection alignment.
How Do Humans Write Code? Large Models Do It the Same Way Too
Long Li (East China Normal University), Liang He (East China Normal University)
CodeAI Code AssistantTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Propose Human-Think Language (HTL), which improves the accuracy of mathematical reasoning by guiding Program-of-Thought (PoT) with complete Chain-of-Thought (CoT) logic, using Focus Attention and reinforcement learning.
How Do Your Code LLMs perform? Empowering Code Instruction Tuning with Really Good Data
Yejie Wang (Beijing University of Posts and Telecommunications), Weiran Xu (Meituan)
CodeData-Centric LearningAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed data cleaning and selection strategies tailored for code instruction tuning, and trained the XCoder model based on these strategies, demonstrating its outstanding performance on multiple code evaluation benchmarks.
How Far Can We Extract Diverse Perspectives from Large Language Models?
Shirley Anugrah Hayati (University of Minnesota), Dongyeop Kang (University of Minnesota)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper studies how to reverse-engineer diverse perspectives from training data using large language models (LLMs), and explores the maximum achievable diversity coverage.
🎯 What it does: By analyzing the training dynamics of NLI samples, automatically divide the test set into three categories: easy, ambiguous, and hard, thereby constructing a more challenging unbiased test set, and using the same method to filter the training set to improve data quality.
How to Leverage Demonstration Data in Alignment for Large Language Model? A Self-Imitation Learning Perspective
Teng Xiao (Pennsylvania State University), Vasant G Honavar (Pennsylvania State University)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Proposes a generic self-imitation learning (GSIL) framework that aligns large language models using offline demonstration data, eliminating adversarial training in traditional RLHF.
I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses
Xuan Ren (University of Adelaide), Lingqiao Liu (University of Adelaide)
CodeData SynthesisData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigate why using LLM-generated responses instead of human-annotated answers can achieve better performance during fine-tuning of the target LLM, and verify the impact of 'familiarity' on learning effectiveness.
🎯 What it does: Fine-tuning the all-mpnet-base-v2 sentence transformer using stance alignment and triplet data to enhance its stance-aware capabilities, thereby improving the effectiveness of opinion mining and stance detection in controversial texts.
I-AM-G: Interest Augmented Multimodal Generator for Item Personalization
Xianquan Wang (University of Science and Technology of China), Qi Liu (University of Science and Technology of China)
CodeGenerationRecommendation SystemTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelMultimodalityRetrieval-Augmented Generation
🎯 What it does: Designed and implemented a user-interest-oriented multimodal personalized generation framework called I-AM-G, which automatically extracts interest tags from user historical interactions and rewrites target item descriptions. Subsequently, it retrieves similar text/images and fuses them through cross-modal attention to guide diffusion models in generating text or images aligned with individual preferences;
🎯 What it does: Designed and implemented an audio watermark model based on a two-stage reversible neural network, separately embedding location codes and information codes to achieve reversible embedding and extraction.
If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions
Reza Esfandiarpoor (Brown University), Stephen Bach
CodeClassificationRepresentation LearningTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: Proposed the EX2 method, which uses reinforcement learning to align the preferences of large language models (LLM) with vision-language models (VLM), generating and analyzing the best conceptual descriptions according to VLM, thereby revealing the text features that VLM relies on when representing concepts.
IFCap: Image-like Retrieval and Frequency-based Entity Filtering for Zero-shot Captioning
Soeun Lee (Hanyang University), Dong-Jin Kim (Hanyang University)
CodeGenerationRetrievalTransformerPrompt EngineeringVision Language ModelContrastive LearningImageVideoTextRetrieval-Augmented Generation
🎯 What it does: Propose a zero-shot image and video captioning method trained using only text data, bridging the gap between text and image modalities through image-like retrieval, fusion module, and frequency-based entity filtering.
Roopal Garg (Google DeepMind), Radu Soricut (Google DeepMind)
CodeGenerationTransformerSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: This paper proposes the ImageInWords framework, generating high-detail, accurate, and hallucination-free image descriptions through human-machine collaborative multi-round annotation;
Improve Student’s Reasoning Generalizability through Cascading Decomposed CoTs Distillation
Chengwei Dai (Institute of Information Engineering, Chinese Academy of Sciences), Songlin Hu (Institute of Information Engineering, Chinese Academy of Sciences)
🎯 What it does: By splitting traditional single-step CoT distillation into two-step cascading learning (first learning the reasoning steps, then learning the answers), answer interference is removed, enhancing the student model's reasoning generalization ability on both in-domain and out-of-domain tasks.
Improving Minimum Bayes Risk Decoding with Multi-Prompt
David Heineman (Georgia Institute of Technology), Wei Xu (Georgia Institute of Technology)
CodeGenerationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a Multi-Prompt MBR decoding scheme, generating candidate sets using multiple prompts and selecting the optimal output by minimizing Bayesian risk (MBR);
🎯 What it does: This paper significantly improves the context independence of phoneme representations by fine-tuning frame-wise phoneme classification on HuBERT, and uses these improved discrete units to train language models.
In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search
Huihan Li (University of Southern California), Xiang Ren (University of Southern California)
CodeGenerationData SynthesisLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes the Logic-Induced Knowledge Search framework (LINK) for systematically generating long-tail reasoning knowledge and constructs the LINT dataset based on this.
CodeComputational EfficiencyLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes the low-cost INDUCT-LEARN framework, which significantly improves the performance of large language models on new tasks by using a small number of input-output examples and a short phrase to induce pseudo-instructions and pseudo-chain-of-thought (CoT).
Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues
Jiao Ou (Kuaishou), Kun Gai (Kuaishou)
CodeGenerationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the IDEAS method, which first inductively extracts instruction strategies from real human-computer dialogues, then uses these strategies to deductively generate diverse and in-depth instructions in new dialogues, constructing multi-round instruction dialogues and using these dialogues to fine-tune chat models.
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
Pengyu Wang (Fudan University), Xipeng Qiu (Fudan University)
CodeSafty and PrivacyTextBenchmarkFinance Related
🎯 What it does: Proposed InferAligner, a simple method that achieves harmless alignment during the inference phase, significantly reducing the model's harmful responses without compromising the performance of downstream tasks.