EMNLP 2024 Papers — Page 5
Conference on Empirical Methods in Natural Language Processing · 1268 papers
ESC-Eval: Evaluating Emotion Support Conversations in Large Language Models
Haiquan Zhao (Shanghai Artificial Intelligence Laboratory), Yingchun Wang (Shanghai Artificial Intelligence Laboratory)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the ESC-Eval framework, leveraging role-playing models and multi-turn dialogue generation to evaluate the performance of large language models in emotional support conversations (ESC), and train an automatic scoring model ESC-RANK based on human annotation.
ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized Transformers
Yuzhe Gu (University of Pennsylvania), Enmao Diao (Duke University)
CompressionTransformerAudio
🎯 What it does: Designed a lightweight full Transformer audio codec ESC, enhancing compression performance through cross-scale residual vector quantization and pre-training.
Estimating Knowledge in Large Language Models Without Generating a Single Token
Daniela Gottesman (Blavatnik School of Computer Science Tel Aviv University), Mor Geva (Blavatnik School of Computer Science Tel Aviv University)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a simple linear detector called KEEN, which predicts the model's knowledge level about an entity by utilizing the internal hidden representations of large language models when processing entity names, thereby estimating answer accuracy and the factualness of generated text without generating any text;
Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works
Xinfeng Yuan (Fudan University), Deqing Yang (Fudan University)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Built and evaluated the ability of large language models (LLMs) to generate character profiles in novels, proposing two assessment tasks based on character profiles: fact consistency detection and motivation recognition.
Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets
Vatsal Gupta (IIT Guwahati), Dan Roth (University of Pennsylvania)
Adversarial AttackTransformerSupervised Fine-TuningPrompt EngineeringTextTabularChain-of-Thought
🎯 What it does: Study the robustness of language models under various input perturbations, propose a multi-set inoculation framework, fine-tune pre-trained models, and apply Chain-of-Thought prompts to large language models, evaluating performance on the Tabular-NLI task.
Evaluating D-MERIT of Partial-annotation on Information Retrieval
Royi Rassin (Amazon Research), Yoav Goldberg (Bar Ilan University)
RetrievalLarge Language ModelTextBenchmark
🎯 What it does: Constructed D-MERIT, a comprehensive retrieval evaluation dataset designed to include all relevant paragraphs for each query, and used this dataset to investigate the impact of partial annotations on retrieval model evaluation.
Evaluating Diversity in Automatic Poetry Generation
Yanran Chen (University of Mannheim), Steffen Eger (University of Technology Nuremberg)
GenerationRecurrent Neural NetworkTransformerLarge Language ModelText
🎯 What it does: This paper systematically evaluates the multi-dimensional diversity (structure, vocabulary, semantics, and rhythm) of various automatic poetry generation models through automated metrics, and explores the relationship between model types, training methods, and diversity.
Evaluating Large Language Models along Dimensions of Language Variation: A Systematik Invesdigatiom uv Cross-lingual Generalization
Niyati Bafna (Johns Hopkins University), David Yarowsky (Johns Hopkins University)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper constructs three Bayesian noise models (phonology/orthography, morphology, lexical), evaluates the performance degradation of LLMs in zero-shot cross-lingual generalization on generated artificial languages, and maps real language distances to noise space through posterior inference to explain and predict performance on low-resource languages.
Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark
Elizabeth Fons (JP Morgan AI Research), Svitlana Vyetrenko (JP Morgan AI Research)
ClassificationRetrievalLarge Language ModelPrompt EngineeringTime SeriesBenchmarkChain-of-Thought
🎯 What it does: Proposes a systematic framework to evaluate the time series understanding capabilities of large language models (LLMs), covering tasks such as feature detection, classification, information retrieval, arithmetic reasoning, and text matching.
Evaluating Large Language Models via Linguistic Profiling
Alessio Miaschi (Istituto di Linguistica Computazionale 'Antonio Zampolli'), Giulia Venturi (Istituto di Linguistica Computazionale 'Antonio Zampolli')
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a sentence generation evaluation method based on morphosyntactic feature constraints to assess the performance of large language models in adhering to specified linguistic attributes.
Evaluating LLMs for Targeted Concept Simplification for Domain-Specific Texts
Sumit Asthana (University of Michigan), Mirella Lapata (Google Deepmind)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Propose a target concept simplification task for domain-specific text to help adult readers understand complex concepts
Evaluating n-Gram Novelty of Language Models Using Rusty-DAWG
William Merrill (New York University), Yanai Elazar (Allen Institute for AI)
GenerationComputational EfficiencyData-Centric LearningTransformerLarge Language ModelTextSequential
🎯 What it does: Built a search tool called RUSTY-DAWG based on CDAWG, which can perform arbitrary-length n-gram matching on large-scale pre-trained corpora (e.g., Pile) in constant time, and used this tool to evaluate the n-gram novelty of language model-generated text;
Evaluating Psychological Safety of Large Language Models
Xingxuan Li (DAMO Academy, Alibaba Group), Lidong Bing (DAMO Academy, Alibaba Group)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper designs bias-free prompts and systematically evaluates and compares the psychological safety and well-being of five mainstream large language models (GPT-3, InstructGPT, GPT-3.5, GPT-4, Llama-2-chat-7B) using psychological assessments (Short Dark Triad, Big Five Inventory, Flourishing Scale, Satisfaction With Life Scale).
Evaluating Readability and Faithfulness of Concept-based Explanations
Meng Li (Renmin University of China), Xiting Wang (University of Science and Technology of China)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper proposes a unified form of concept explanation and provides two automated evaluation metrics: faithfulness and readability. Based on this, a meta-evaluation framework grounded in measurement theory is constructed to assess the reliability and validity of these metrics.
Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Models
Yi Zhou (Cardiff University), Jose Camacho-Collados (Cardiff University)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextTime SeriesBenchmark
🎯 What it does: Studied the changes in social bias over time for masked language models trained on social media data, systematically evaluating temporal fluctuations in bias using the AULA metric on CrowS-Pairs and StereoSet.
Evaluating the Effectiveness of Large Language Models in Establishing Conversational Grounding
Biswesh Mohapatra (Inria), Justine Cassell (Inria)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed an evaluation benchmark for dialogue grounding, using perplexity and prompt selection as two automated methods to evaluate the performance of multiple LLMs in different grounding scenarios (e.g., repair, cancellation, request for repair), and further analyzed model differences through embedding distance analysis and positive-negative reward fine-tuning.
Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection
Zekun Li (University of California Santa Barbara), Xifeng Yan (University of California Santa Barbara)
Safty and PrivacyAdversarial AttackTransformerSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Established a benchmark to evaluate the instruction-following robustness of large language models under prompt injection attacks, using QA datasets for quantitative analysis.
EVEDIT: Event-based Knowledge Editing for Deterministic Knowledge Propagation
Jiateng Liu (University of Illinois Urbana Champaign), Heng Ji (University of Illinois Urbana Champaign)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose the event-driven knowledge editing framework EVEDIT to address the uncertainty issues caused by traditional triplet editing
Event Causality Identification with Synthetic Control
Haoyu Wang (University of Pennsylvania), Kyle Richardson (Allen Institute for AI)
ClassificationData SynthesisRetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Proposes a text event causality identification method based on synthetic control, utilizing retrieval and synthesis of 'twin' control units in zero-shot scenarios to determine causality between event pairs.
Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering
Sungho Ko (Yonsei University), Dongha Lee (Yonsei University)
Knowledge DistillationRepresentation LearningTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextGraphRetrieval-Augmented Generation
🎯 What it does: This study proposes an evidence-focused factual summarization framework named EFSUM to enhance zero-shot question answering systems based on knowledge graphs;
Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently
Kanishka Misra (University of Texas at Austin), Kyle Mahowald (University of Texas at Austin)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Systematically evaluate the performance of language models on attribute inheritance reasoning tasks by incorporating experimental context (example prompts and instructions) into the COMPS dataset.
Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM
Haw-Shiuan Chang (University of Massachusetts Amherst), Tagyoung Chung (Amazon AGI Foundations)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Propose and theoretically analyze the working mechanism of contrastive decoding (CD), discovering its equivalence to linear extrapolation of a maximum hypothesis LM, and on this basis propose Asymptotic Probability Decoding (APD). By jointly training a small LM with a probability curve fitter, APD achieves more robust extrapolation in the probability space, thereby improving the factual accuracy and reasoning accuracy of generated text.
Explicit Memory Learning with Expectation Maximization
Zhangyue Yin (Fudan University), Xuanjing Huang (Fudan University)
OptimizationTransformerLarge Language ModelTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed an explicit memory update framework called EM² based on the Expectation-Maximization (EM) algorithm, aiming to enable large language models to continuously learn and improve their external text memory in sequential reasoning tasks;
Explicit, Implicit, and Scattered: Revisiting Event Extraction to Capture Complex Arguments
Omar Sharif (Dartmouth College), Sarah M. Preum (Dartmouth College)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the DiscourseEE dataset and reformulate the event extraction task as text generation, supporting the extraction of explicit, implicit, and distributed arguments.
EXPLORA: Efficient Exemplar Subset Selection for Complex Reasoning
Kiran Purohit (IIT Kharagpur), Avishek Anand (TU Delft)
OptimizationComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the EXPLORA algorithm for static exemplar subset selection in complex reasoning tasks, thereby improving the inference performance of large language models (LLMs).
Exploring Intra and Inter-language Consistency in Embeddings with ICA
Rongzhi Li (University of Tokyo), Hitomi Yanaka (University of Tokyo)
Explainability and InterpretabilityRepresentation LearningText
🎯 What it does: Investigated the reproducibility of independent components within monolingual word vectors and their cross-lingual correspondences, using Icasso to verify the stability of independent components within the same language and statistical tests to confirm the consistency of semantic axes across languages.
Exploring Intrinsic Language-specific Subspaces in Fine-tuning Multilingual Neural Machine Translation
Zhe Cao (Nara Institute of Science and Technology), Taro Watanabe (Nara Institute of Science and Technology)
TransformerSupervised Fine-TuningText
🎯 What it does: Investigated the intrinsic low-rank subspaces of each language in multilingual neural machine translation models during fine-tuning, proposing language-specific LoRA (LSLo) to efficiently separate subspaces and combine gradient progressive pruning and architecture learning for parameter-efficient fine-tuning.
Exploring Nested Named Entity Recognition with Large Language Models: Methods, Challenges, and Insights
Hongjin Kim, Harksoo Kim
RecognitionTransformerLarge Language ModelPrompt EngineeringTextBiomedical Data
🎯 What it does: The study explores the implementation of nested named entity recognition on large language models (LLMs), investigating various output formats, reasoning techniques, and instruction tuning strategies;
Exploring Space Efficiency in a Tree-based Linear Model for Extreme Multi-label Classification
He-Zhe Lin (National Taiwan University), Chih-Jen Lin (National Taiwan University)
ClassificationText
🎯 What it does: Study the space efficiency of tree-based linear models in extreme multi-label text classification under sparse data conditions, and provide a method for estimating model size before training.
Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems
Jun Zhao (Fudan University), Xuanjing Huang (Fudan University)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Investigated the compositional generalization ability of large language models (LLMs) in mathematical reasoning, proposed a novel trap-style problem set called MATHTRAP, and experimentally evaluated model performance in handling logical traps.
Exploring the Learning Capabilities of Language Models using LEVERWORLDS
Eitan Wagner (Hebrew University of Jerusalem), Omri Abend (Hebrew University of Jerusalem)
Data SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextTabularPhysics Related
🎯 What it does: This paper designs a generable and measurable physical experiment framework called LEVERWORLDS, which generates synthetic data based on the principles of lever physics. Subsequently, the paper conducts experiments comparing the sample efficiency and predictive accuracy of Transformer models (OPT, GPT series) with classical statistical learning methods (MLE, Logistic Regression) on this dataset, and explores the possibility of collaboration between LLMs and classical models in In-Context Learning and Pipeline schemes.
Exploring the Practicality of Generative Retrieval on Dynamic Corpora
Chaeeun Kim (KAIST AI), Minjoon Seo (KAIST AI)
RetrievalTransformerTime SeriesBenchmark
🎯 What it does: Evaluate and compare the adaptability, robustness, and efficiency of generative retrieval and dual-encoder retrieval under dynamic corpora, proposing the DynamicIR framework.
Exploring the Role of Reasoning Structures for Constructing Proofs in Multi-Step Natural Language Reasoning with Large Language Models
Zi’ou Zheng, Xiaodan Zhu (Queen's University)
Graph Neural NetworkTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper improves the proof construction process in multi-step natural language reasoning by incorporating structured examples and structured pruning into the prompts of large language models.
Exploring Union and Intersection of Visual Regions for Generating Questions, Answers, and Distractors
Wenjian Ding (Nankai University), Zhenglu Yang (Nankai University)
GenerationRecurrent Neural NetworkTransformerLarge Language ModelVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose a unified framework named ReBo for generating visual multiple-choice questions, answers, and distractors (QAD), leveraging a cyclic multimodal encoder and combining union and intersection scores of image regions to control the diversity and coverage of generated content.
Extending Context Window of Large Language Models from a Distributional Perspective
Yingsheng Wu (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose a context window expansion strategy based on rotation angle distribution, extending the available context length of RoPE by minimizing rotation angle distribution perturbation in each dimension.
External Knowledge-Driven Argument Mining: Leveraging Attention-Enhanced Multi-Network Models
Debela Gemechu (University of Dundee), Chris Reed (University of Dundee)
ClassificationTransformerLarge Language ModelContrastive LearningTextBiomedical DataRetrieval-Augmented Generation
🎯 What it does: This paper searches for semantic paths between concepts in dictionaries (WordNet, ConceptNet) and semi-structured resources (Wikipedia), and uses an attention-enhanced multi-network architecture (Siamese and Triplet networks) to predict relationships (support, conflict, or unrelated) between argument units (ADU) in dialogues.
Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction
Bowen Zhang (National University of Singapore), Harold Soh (National University of Singapore)
TransformerLarge Language ModelPrompt EngineeringTextGraphRetrieval-Augmented Generation
🎯 What it does: Proposes a three-stage framework EDC (Extract-Define-Canonicalize), decomposing knowledge graph construction into open-information extraction, schema definition, and schema canonicalization, while introducing Schema Retriever for retrieval enhancement;
Extracting Prompts by Inverting LLM Outputs
Collin Zhang, Vitaly Shmatikov
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposes a black-box method without logits or adversarial queries, named output2prompt, which infers hidden prompts from LLM outputs.
Eyes Don’t Lie: Subjective Hate Annotation and Detection with Gaze
Özge Alacam, Sina Zarrieß (Bielefeld University)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextMultimodality
🎯 What it does: Constructed the GAZE4HATE dataset containing annotators' eye movement records, subjective hate severity ratings, and annotation rationales, and investigated the predictive role of eye movement features in hate detection; subsequently designed and evaluated MEANION, the first hate speech detection model integrating eye movement information.
F^2RL: Factuality and Faithfulness Reinforcement Learning Framework for Claim-Guided Evidence-Supported Counterspeech Generation
Haiyang Wang (National University of Defense Technology), Bin Zhou (National University of Defense Technology)
GenerationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Proposed the F RL framework, which first generates and self-evaluates counterarguments against hate speech to select the most suitable ones, then retrieves evidence related to the counterarguments through a coarse-to-fine retrieval process, and finally generates hate speech rebuttals based on the counterarguments and evidence using reinforcement learning.
FAC^2E: Better Understanding Large Language Model Capabilities by Dissociating Language and Cognition
Xiaoqiang Wang (Université de Montréal), Bang Liu (Université de Montréal)
Explainability and InterpretabilityLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Proposed the FAC E 2 framework, which adopts a four-dimensional system separating language and cognition (language knowledge, formal knowledge, world modeling, social modeling), decomposing each dimension into three steps: recalling knowledge, utilizing knowledge, and problem-solving, enabling fine-grained and interpretable evaluation of LLMs.
Factuality of Large Language Models: A Survey
Yuxia Wang (MBZUAI), Preslav Nakov (MBZUAI)
TransformerTextMultimodalityReview/Survey PaperBenchmarkRetrieval-Augmented Generation
🎯 What it does: Summarize and systematically evaluate the factuality of large language models, organize assessment and improvement methods at each stage from pre-training to post-processing, and explore factuality issues in multimodal models.
Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments
Han Zhou (University of Cambridge), Anna Korhonen (University of Cambridge)
Explainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper addresses the sensitivity and bias issues of large language models (LLMs) in evaluation tasks by proposing a zero-shot fairness optimization framework (ZEPO), which enhances the consistency between evaluation results and human judgments through unsupervised fairness maximization on instructions.
FairFlow: Mitigating Dataset Biases through Undecided Learning for Natural Language Understanding
Jiali Cheng (University of Massachusetts Lowell), Hadi Amiri (University of Massachusetts Lowell)
Domain AdaptationRepresentation LearningData-Centric LearningTransformerContrastive LearningText
🎯 What it does: Propose a multi-view contrastive learning framework named FAIRFLOW, which generates biased views by utilizing data and model perturbations, and enables the model to maintain 'undecided' predictions on these views to eliminate data bias;
FAME: Towards Factual Multi-Task Model Editing
Li Zeng (Beijing Institute of Technology), Yuhang Guo (Beijing Institute of Technology)
Data-Centric LearningTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes a model editing task oriented toward practical applications, constructs a multi-task dataset named FAME containing 128k real facts, and introduces the SKEME editing method based on this dataset.
Fast Forwarding Low-Rank Training
Adir Rahamim (Technion - Israel Institute of Technology), Yonatan Belinkov (Technion - Israel Institute of Technology)
OptimizationComputational EfficiencyLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Propose the Fast Forward method, which accelerates training by using line search to repeatedly move along the recent gradient direction on a small validation set within low-rank fine-tuning (LoRA/DoRA);
FEDKIM: Adaptive Federated Knowledge Injection into Medical Foundation Models
Xiaochen Wang (Pennsylvania State University), Fenglong Ma (Georgia State University)
Federated LearningSafty and PrivacyTransformerSupervised Fine-TuningMixture of ExpertsMultimodalityBiomedical Data
🎯 What it does: Under the federated learning framework, the FEDKIM method is proposed, which extracts knowledge from private medical data through a lightweight local model and injects the extracted multi-modal multi-task knowledge into a medical base model on the server.
Fewer is More: Boosting Math Reasoning with Reinforced Context Pruning
Xijie Huang (Hong Kong University of Science and Technology), Mao Yang (Hong Kong University of Science and Technology)
Computational EfficiencyTransformerLarge Language ModelReinforcement LearningTextBenchmarkChain-of-Thought
🎯 What it does: Designed and implemented the CoT-Influx plugin, utilizing a two-step pruning approach (from coarse to fine) to remove redundant CoT examples and tokens, maximizing input volume to enhance LLM mathematical reasoning performance.
FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
Ajay Kumar Jaiswal (University of Texas at Austin), Aditya Akella (University of Oxford)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: The study skips redundant FFN blocks during autoregressive large language model inference, avoiding KV cache conflicts and achieving inference acceleration.
Fill In The Gaps: Model Calibration and Generalization with Synthetic Data
Yang Ba (Arizona State University), Rong Pan (Arizona State University)
ClassificationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Generate synthetic text targeting calibration gaps using large language models to improve model accuracy and calibration error (ECE).
Filtered Direct Preference Optimization
Tetsuro Morimura, Kaito Ariu (CyberAgent)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Investigated the impact of text quality on direct preference optimization (DPO) and proposed filtered DPO (fDPO), which leverages a reward model to filter low-quality samples.
Finding Blind Spots in Evaluator LLMs with Interpretable Checklists
Sumanth Doddapaneni (Nilekani Centre at AI4Bharat), Mitesh M Khapra
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes the FBI framework, which evaluates the capability of LLMs in text generation tasks by constructing targeted perturbation checks, focusing on factual accuracy, instruction following, long-text coherence, and reasoning ability.
FinDVer: Explainable Claim Verification over Long and Hybrid-content Financial Documents
Yilun Zhao (Yale NLP), Arman Cohan (Yale NLP)
Explainability and InterpretabilityTransformerLarge Language ModelTextTabularBenchmarkFinance RelatedRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the FINDVER benchmark for evaluating the interpretability claim verification of LLMs in long-form financial documents containing text and tables.
Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary Debates
Aida Kostikova (Bielefeld University), Steffen Eger (University of Mannheim)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningText
🎯 What it does: This study constructs a fine-grained cohesion and anti-cohesion annotation dataset for German parliamentary debate texts from 1867 to 2022, leveraging large language models such as GPT-4 to achieve high-quality automated annotation, revealing the temporal evolution of cohesion types in discussions about women and immigration topics;
Fine-grained Pluggable Gradient Ascent for Knowledge Unlearning in Language Models
XiaoHua Feng, Zibin Lin (Hangzhou Dianzi University)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a Fine-Grained Pluggable Gradient Ascent (FPGA) method to achieve efficient forgetting of sensitive knowledge in language models while minimizing negative impacts on general model capabilities.
Fine-Grained Prediction of Reading Comprehension from Eye Movements
Omer Shubi (Technion Israel Institute of Technology), Yevgeni Berzak (Technion Israel Institute of Technology)
ClassificationConvolutional Neural NetworkRecurrent Neural NetworkTransformerLarge Language ModelTextMultimodalityTime Series
🎯 What it does: This study predicts the accuracy of single-paragraph single-question reading comprehension using eye movement data, proposing and systematically evaluating three Transformer-based text-eye movement fusion models.
Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
Dilara Soylu (Stanford University), Omar Khattab (Stanford University)
OptimizationHyperparameter SearchTransformerSupervised Fine-TuningPrompt EngineeringMultimodality
🎯 What it does: Proposed an algorithm called BetterTogether, which alternates between prompt optimization and weight fine-tuning to enhance the final task performance of multi-modal language model pipelines.
Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice?
Dawei Zhu (Saarland University), Dietrich Klakow (Saarland University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Investigated supervised fine-tuning (SFT) on large language models to enhance machine translation capabilities, systematically evaluated the impact of training data volume, single translation direction, and synthetic noise on translation performance.
Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs
Oded Ovadia (Microsoft), Oren Elisha (Microsoft)
TransformerSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Compared the effectiveness of two knowledge injection methods—unsupervised fine-tuning (FT) and retrieval-augmented generation (RAG)—on multiple knowledge-intensive tasks.
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension
Junzhuo Liu (University of Electronic Science and Technology of China), Peng Wang (University of Electronic Science and Technology of China)
RecognitionLarge Language ModelSupervised Fine-TuningDiffusion modelMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the FineCops-Ref dataset to evaluate the understanding of fine-grained compositional referring expressions using fine-grained difficulty levels and negative samples.
Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models
Jeonghwan Kim (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
RecognitionExplainability and InterpretabilityPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper analyzes the shortcomings of large vision-language models optimized with instruction tuning in fine-grained visual classification tasks, and proposes the attribute-based multi-grained benchmark FINER and the ATTRSEEK prompting scheme to enhance zero-shot recognition and interpretability performance.
First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model Reasoning
Yoichi Aoki (Tohoku University), Kentaro Inui (MBZUAI)
Explainability and InterpretabilityTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Study the dynamic usage strategies of language models in multi-step reasoning processes, involving shallow heuristics and rational reasoning, and verify through controlled experiments that models initially use heuristics and gradually shift to goal-oriented reasoning.
FIRST: Faster Improved Listwise Reranking with Single Token Decoding
Revanth Gangi Reddy (University of Illinois Urbana Champaign), Heng Ji (University of Illinois Urbana Champaign)
RetrievalComputational EfficiencyTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes a list-based reranker based on single-token decoding of LLMs, named FIRST.
FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation
KaShun Shum, Muhammad Omer Raza (Purdue University)
Knowledge DistillationTransformerLarge Language ModelText
🎯 What it does: Propose an efficient and trustworthy distillation method called FIRST, which improves the accuracy and calibration of small models by leveraging the teacher model's 'concentrated knowledge' and temperature regulation.
Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models
Ji Liu (HiThink Research), Dejing Dou (Inria)
Federated LearningComputational EfficiencyTransformerSupervised Fine-TuningText
🎯 What it does: Studied efficient fine-tuning of large language models in federated learning environments, and proposed FibecFed, an efficient curriculum learning and sparse parameter update framework based on Fisher information.
Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models
Sander Land (Cohere), Max Bartolo (Cohere)
Anomaly DetectionTransformerLarge Language ModelText
🎯 What it does: This paper proposes an automatic method to detect 'untrained/under-trained' tokens (glitch tokens) in large language models, along with an open-source tool.
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document
Joonho Yang (Chung-Ang University), Hwanhee Lee (Chung-Ang University)
Explainability and InterpretabilityTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the FIZZ system, which detects factual consistency in abstract summaries by leveraging core citation parsing, LLM-generated atomic facts, NLI scoring, and adaptive granularity expansion.
Flee the Flaw: Annotating the Underlying Logic of Fallacious Arguments Through Templates and Slot-filling
Irfan Robbani (JAIST), Kentaro Inui (MBZUAI)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Developed a logic fallacy structure annotation scheme based on templates and slot filling, and constructed the FtF dataset containing 400 LOGIC arguments.
FlipGuard: Defending Preference Alignment against Update Regression with Constrained Optimization
Mingye Zhu (University of Science and Technology of China), Zhendong Mao (University of Science and Technology of China)
OptimizationKnowledge DistillationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposes the FlipGuard framework, which detects and mitigates update regression by introducing reward-based focus constraints in preference alignment.
FLIRT: Feedback Loop In-context Red Teaming
Ninareh Mehrabi (Amazon AGI Foundations), Rahul Gupta (Amazon AGI Foundations)
Adversarial AttackTransformerLarge Language ModelPrompt EngineeringImageText
🎯 What it does: Propose an automated red teaming framework called FLIRT, which utilizes context learning and feedback loops to iteratively generate adversarial prompts aimed at inducing unsafe outputs from text-to-image and text-to-text models.
Focused Large Language Models are Stable Many-Shot Learners
Peiwen Yuan (Beijing Institute Of Technology), Kan Li (Beijing Institute Of Technology)
Hyperparameter SearchTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Studied the problem where increasing the number of demonstrations in multi-example context learning leads to attention dispersion and performance degradation, and proposed an untrained focus attention method called FOCUSICL to stabilize multi-example learning.
FOLIO: Natural Language Reasoning with First-Order Logic
Simeng Han (Yale University), Dragomir Radev (Yale University)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed a high-quality dataset named FOLIO containing natural language reasoning and first-order logic (FOL) annotations, and proposed two tasks: natural language reasoning and FOL translation; conducted benchmark experiments on multiple models using this dataset.
FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture
Wenyan Li (University of Copenhagen), Desmond Elliott (University of Copenhagen)
Large Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: This paper proposes the FoodieQA multimodal dataset, which conducts fine-grained multi-image, single-image, and text question answering benchmarking for Chinese food culture.
FOOL ME IF YOU CAN! An Adversarial Dataset to Investigate the Robustness of LMs in Word Sense Disambiguation
Mohamad Ballout (University of Osnabrück), Kai-Uwe Kühnberger (University of Osnabrück)
ClassificationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Constructed a coarse-grained word sense disambiguation (WSD) dataset named FOOL, and evaluated the robustness of various language models in both positive and adversarial contexts.
Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting
Maxime Kayser (University of Oxford), Oana-Maria Camburu (University College London)
Explainability and InterpretabilityConvolutional Neural NetworkTransformerVision Language ModelMultimodalityBiomedical Data
🎯 What it does: In the clinical decision support system for chest X-ray diagnosis, the authors evaluated three explainability methods through a large-scale (85 medical professionals) user experiment: visual saliency maps, natural language explanations (NLE), and their combination, and examined the impact of the correctness of AI recommendations and explanations themselves on user performance.
Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models
Xinyu Liu (Northeastern University), JingBo Zhu
Explainability and InterpretabilityComputational EfficiencyRecurrent Neural NetworkTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed the Forgetting Curve method, evaluating the memory capacity of long-context language models by comparing replication tasks with language model accuracy curves, and conducting systematic experiments on multiple models.
Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge
Jiahuan Li (Nanjing University), Jiajun Chen (Nanjing University)
Data SynthesisExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper systematically studies the learning preferences of large language models when faced with data containing conflicting knowledge, regarding different text features (such as writing style, spelling accuracy), and proposes the 'consistency-driven feature preference hypothesis,' explaining that models determine which knowledge to learn by identifying the consistency of the text with the majority data.
Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation
Tu Vu (Google DeepMind), Yun-Hsuan Sung (Google DeepMind)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Collect and unify publicly available human evaluation data, constructing over 100 quality assessment tasks (totaling 5.3M human judgments), and perform multi-task supervised training on PaLM-2-24B to generate a general-purpose automatic evaluation model FLAMe and its refined versions (FLAMe-RM, FLAMe-Opt-RM).
Free your mouse! Command Large Language Models to Generate Code to Format Word Documents
Shihao Rao (Chinese Academy of Sciences), Can Ma (Chinese Academy of Sciences)
AI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes an automatic Word document formatting method based on LLM called TEXT-TO-FORMAT, and constructs an evaluation dataset named DOCFORMEVAL.
FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs
Yiyuan Li (UNC-Chapel Hill), Pengfei Liu (Shanghai Jiao Tong University)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose and construct the FROG benchmark to evaluate the performance of large language models on fuzzy reasoning tasks involving generalized quantifiers.
From Bottom to Top: Extending the Potential of Parameter Efficient Fine-Tuning
Jihao Gu (Beijing University of Posts and Telecommunications), Ping Gong (Beijing University of Posts and Telecommunications)
OptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed a hybrid LoRA-Prefix tuning method (HLPT) and a half-layer tuning method (H2LPT), which allocate different parameter-efficient fine-tuning techniques to different layers of large language models, and conducted experiments on models such as LLaMA and GPT-J.
From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment
Yusuke Hirota (Osaka University), Yuta Nakashima (NVIDIA Research)
GenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: This paper systematically evaluates the negative impacts of using large language models (LLM) for image caption enhancement (GCE) on gender bias and hallucinations, and further investigates whether these issues are amplified after model training.
From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP
Marius Mosbach (Mila Quebec Ai Institute), Mor Geva (Tel Aviv University)
Explainability and InterpretabilitySupervised Fine-TuningTextReview/Survey Paper
🎯 What it does: This paper quantifies and evaluates the impact of interpretability and analysis (IA) research on the NLP field by constructing a citation network containing over 185K ACL/EMNLP papers and conducting a questionnaire survey with 138 NLP researchers.
From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking
Siyuan Wang (University of Southern California), Zhongyu Wei (Alibaba Inc)
Safty and PrivacyAdversarial AttackLarge Language ModelMultimodalityReview/Survey PaperBenchmark
🎯 What it does: This paper systematically reviews jailbreak research on LLMs and MLLMs, summarizes evaluation datasets, attack methods, and defense strategies, and conducts an in-depth analysis of the current status and challenges of multi-modal jailbreaks.
From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models
Mehar Bhatia (University of British Columbia), Vered Shwartz (University of British Columbia)
Object DetectionRetrievalTransformerVision Language ModelImageTextBenchmark
🎯 What it does: Proposed the GLOBALRG benchmark, which includes two tasks: multicultural image retrieval (cross-universal retrieval) and visual localization of culture-specific concepts (cultural visual localization).
From RAG to Riches: Retrieval Interlaced with Sequence Generation
Palak Jain (Google Deepmind), Tom Kwiatkowski (Google Deepmind)
GenerationRetrievalLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose a RICHES framework that interweaves retrieval and text generation during a single LLM decoding process.
From the Least to the Most: Building a Plug-and-Play Visual Reasoner via Data Synthesis
Chuanqi Cheng (Renmin University of China), Rui Yan (Renmin University of China)
Data SynthesisLarge Language ModelSupervised Fine-TuningImageTextMultimodalityChain-of-Thought
🎯 What it does: By introducing the least-to-most visual reasoning paradigm and bottom-up data synthesis methods, a 50k visual reasoning dataset (VIREO) was constructed, and a pluggable visual reasoner was fine-tuned on this dataset, significantly enhancing the multi-step reasoning capabilities of existing vision-language models (VLMs).
Fuse to Forget: Bias Reduction and Selective Memorization through Model Fusion
Kerem Zaman (UNC Chapel Hill), Shashank Srivastava (UNC Chapel Hill)
Safty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelContrastive LearningText
🎯 What it does: This paper explores the mechanism of model fusion (weight averaging) on knowledge retention and forgetting in language models, verifying its effectiveness in eliminating irrelevant shortcuts, social biases, and overfitting without requiring additional annotations or retraining.
FuseGen: PLM Fusion for Data-generation based Zero-shot Learning
Tianyuan Zou (Tsinghua University), Ya-Qin Zhang (Tsinghua University)
GenerationData SynthesisTransformerLarge Language ModelText
🎯 What it does: Propose the FuseGen framework, which leverages collaboration among multiple pre-trained language models (PLMs) to generate high-quality synthetic data. By integrating cross-model evaluation and self-improving weight adjustment, it trains a small task-specific model (STM), achieving zero-shot learning and significantly enhancing STM performance.
GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities
Sreyan Ghosh (University of Maryland), Dinesh Manocha (University of Maryland)
ClassificationRecognitionGenerationTransformerLarge Language ModelPrompt EngineeringTextMultimodalityAudio
🎯 What it does: Proposed GAMA, a general large audio-language model integrating multiple audio features (Audio Q-Former, AST + multi-layer aggregator) and LLM, achieving complex reasoning capabilities through the synthesized CompA-R dataset;
Game on Tree: Visual Hallucination Mitigation via Coarse-to-Fine View Tree and Game Theory
Xianwei Zhuang (Peking University), Yuexian Zou (Peking University)
Explainability and InterpretabilityComputational EfficiencyVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose a training-agnostic, plug-and-play decoding strategy called GTHM to alleviate visual hallucinations in large vision-language models.
GDPO: Learning to Directly Align Language Models with Diversity Using GFlowNets
Oh Joon Kwon (KAIST AI), Kee-Eung Kim (KAIST AI)
Reinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningFlow-based ModelText
🎯 What it does: Aligning language models with diversity-oriented alignment using GFlowNet on offline preference data
GDTB: Genre Diverse Data for English Shallow Discourse Parsing across Modalities, Text Types, and Domains
Yang Janet Liu (MaiNLP, Center for Information and Language Processing, LMU Munich), Amir Zeldes (Georgetown University)
ClassificationData SynthesisDomain AdaptationData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Based on the eRST structures annotated in the GUM corpus, we constructed GDTB — a dataset covering 16 English spoken and written discourse domains, containing approximately 13.6k shallow discourse relations annotated in the PDTB v3 style.
Generalizing Clinical De-identification Models by Privacy-safe Data Augmentation using GPT-4
Woojin Kim (Korean National Police Agency), Jaejin Lee (Seoul National University)
Data SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringBiomedical DataElectronic Health Records
🎯 What it does: Generate synthetic clinical records without PHI using GPT-4 via one-shot/zero-shot prompts, and train BERT-like models on these records to enhance the generalization performance of cross-dataset de-identification models.
Generate-on-Graph: Treat LLM as both Agent and KG for Incomplete Knowledge Graph Question Answering
Yao Xu (University of Chinese Academy of Sciences), Kang Liu (University of Chinese Academy of Sciences)
GenerationTransformerLarge Language ModelAgentic AIPrompt EngineeringTextGraphRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes a new task called IKGQA, which uses large language models (LLMs) for question answering under incomplete knowledge graphs (IKGs), and constructs a corresponding dataset; simultaneously, it introduces the Generate-on-Graph (GoG) method, which employs LLMs as both an Agent and a KG, exploring and generating missing facts within the knowledge graph through a Thinking-Searching-Generating framework to address missing knowledge issues.
Generating Demonstrations for In-Context Compositional Generalization in Grounded Language Learning
Sam Spilsbury (Aalto University), Alexander Ilin (Aalto University)
GenerationMeta LearningTransformerPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: In language learning tasks with environmental state awareness, a method called DemoGen was designed to dynamically generate suitable support examples for each query using an autoregressive language model and Bootstrap Transformer, and solve test problems through in-context learning.
Generation with Dynamic Vocabulary
Yanting Liu (East China Normal University), Xiaoling Wang (East China Normal University)
GenerationDomain AdaptationTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Propose a dynamically expandable vocabulary language model framework that uses a trainable phrase encoder to treat any text fragment as a single generation unit, enabling the insertion of dynamic phrases during generation.
Generative Models for Automatic Medical Decision Rule Extraction from Text
Yuxin He (Harbin Institute of Technology), Xiaoling Wang (East China Normal University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Built a generative model capable of automatically extracting medical decision trees (in binary tree form) from medical text, employing both sequence-to-sequence and autoregressive paradigms.
Generative Subgraph Retrieval for Knowledge Graph–Grounded Dialog Generation
Jinyoung Park (Korea University), Hyunwoo J. Kim (Korea University)
GenerationRetrievalTransformerLarge Language ModelGraphRetrieval-Augmented Generation
🎯 What it does: Designed a dialogue generation model called DialogGSR based on generative subgraph retrieval, which can directly generate knowledge graph subgraph sequences on language models and integrate them with dialogue history to generate responses.
GENRA: Enhancing Zero-shot Retrieval with Rank Aggregation
Georgios Katsimpras (NCSR Demokritos), Georgios Paliouras (NCSR Demokritos)
RetrievalTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Designed a zero-shot retrieval pipeline GENRA, which leverages LLM to generate multiple prompt sentences, first performs query expansion and retrieval, then uses LLM for relevance assessment, and finally conducts multi-document retrieval on each validated document and obtains the final ranking through rank aggregation.