Conference on Empirical Methods in Natural Language Processing · 380 papers
Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?
Cheng Zhang (Imperial College London), Yiren Zhao (Imperial College London)
CodeComputational EfficiencyHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: To address low-precision inference for large language models, this paper proposes a block-level quantization scheme that utilizes shared exponents to reduce scale shift, achieving near-lossless inference at 6-bit and 4-bit precisions.
Peng Fu (Zhejiang University), Junbo Zhao (Zhejiang University)
CodeRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextGraphReview/Survey PaperBenchmark
🎯 What it does: Systematically evaluate existing knowledge injection frameworks, finding that alignment injection performs almost equally to random or noise injection, with random even being superior; further analyze the reasons, propose using cleaned conceptual knowledge as the injection source, and verify that this method can significantly restore the advantages of alignment injection.
S2abEL: A Dataset for Entity Linking from Scientific Tables
Yuze Lou (University of Michigan), Doug Downey (Allen Institute for AI)
CodeRetrievalTransformerLarge Language ModelTabularBenchmark
🎯 What it does: Created the first entity linking dataset for scientific tables (especially machine learning experiment result tables) called S2abEL, and designed a complete baseline model for table entity linking that supports detecting and annotating out-KB entities even when the knowledge base is incomplete.
🎯 What it does: Proposed and implemented SCANDL, a discrete sequence-to-sequence (seq2seq) generation framework based on diffusion models, for synthesizing human eye movement scan paths given text.
🎯 What it does: Design and release the SciRepEval benchmark, which includes 24 multi-format tasks (classification, regression, proximity retrieval, retrieval), and use this benchmark to evaluate and improve scientific document representation models; based on this, propose the SPECTER2 multi-format model, which uses control codes and adapters to achieve task-format specific embeddings.
Seeing through the mess: evolutionary dynamics of lexical polysemy
Andreas Baumann (University of Vienna), Benjamin Roth (University of Vienna)
CodeTransformerTextOrdinary Differential Equation
🎯 What it does: This study systematically analyzes the evolutionary mechanisms of polysemy by combining a mathematical model based on adaptive dynamics with empirical testing, focusing on the impact of word frequency, non-conformist behavior, and semantic discriminability on semantic differentiation, and verifying the model's predictions on historical English data.
Select, Prompt, Filter: Distilling Large Language Models for Summarizing Conversations
Minh-Quang Pham (Zoom Video Communications), Marco Turchi (Zoom Video Communications)
CodeGenerationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Knowledge distillation for large language models (e.g., ChatGPT) using a three-step process (selection, prompting, filtering) to generate high-quality dialogue summary data, which is then used to fine-tune a small generative model (BART-large) for forum and email conversations.
CodeRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Propose the SELF-ICL framework, enabling zero-shot self-generated demonstrations and ICL based solely on test inputs and task descriptions.
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Potsawee Manakul (University of Cambridge), Mark Gales
CodeAnomaly DetectionTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose SelfCheckGPT, a zero-resource, black-box method for detecting hallucinations in generative large language models, which determines the authenticity of sentences by measuring consistency across multiple sampled outputs.
Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models
Junpeng Li (National Key Laboratory of General Artificial Intelligence), Zilong Zheng (National Key Laboratory of General Artificial Intelligence)
CodeGenerationData SynthesisData-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Propose a semi-automated data augmentation framework that combines large language models (LLMs) with natural language inference (NLI) modules to automatically generate high-quality relation triplets for the document-level relation extraction (DocRE) task, and construct an improved test set DocGNRE and an expanded training set.
🎯 What it does: Proposed the SKD-NER model, achieving continual learning for named entity recognition (NER) that balances learning of both new and existing entity categories.
Skill-Based Few-Shot Selection for In-Context Learning
Shengnan An (Xi'an Jiaotong University), Jian-Guang Lou (Microsoft Corporation)
CodeRetrievalMeta LearningLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Propose SKILL-KNN, a training-free few-shot selection method that generates skill descriptions through prompting and performs KNN retrieval based on these descriptions, aiming to enhance the contextual learning effectiveness of large language models.
SLOG: A Structural Generalization Benchmark for Semantic Parsing
Bingzhi Li (Universite Paris Cite), Najoung Kim (Boston University)
CodeData SynthesisTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed the SLOG dataset, specifically designed to evaluate the performance of semantic parsing models on structural generalization (new syntactic structures), expanding the existing COGS benchmark;
SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts
Joon-Young Choi (Korea University), SangKeun Lee (Korea University)
CodeComputational EfficiencyTransformerPrompt EngineeringMixture of ExpertsTextBenchmark
🎯 What it does: Propose the SMoP (Sparse Mixture-of-Prompts) method, which achieves efficient prompt tuning by using extremely short soft prompts through sparse mixing of prompts.
CodeTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes a method for cross-cultural social norm discovery and comparison, utilizing the Chinese Zhihu platform and the American SOCIALCHEMISTRY dataset. By employing cross-lingual contextual alignment, few-shot prompt-based rule extraction, and chain-of-thought reasoning, it generates interpretable text entailment pairs, constructing a dataset of 3,069 cross-cultural social norm comparisons between China and the U.S.
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose the SOUL task, which includes two subtasks: Review Comprehension (RC) and Justification Generation (JG), to evaluate models' capabilities in sentiment understanding and reasoning.
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4
Kent K. Chang (University of California, Berkeley), David Bamman (University of California, Berkeley)
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper systematically evaluates the memory capacity of ChatGPT and GPT-4 for 571 novels using 'name-filling' membership inference queries, analyzes the impact of memory bias on downstream task performance, conducts benchmark comparisons with models like BERT, and proposes the use of models with publicly available training data to improve interpretability and reproducibility.
Speech Recognition and Meaning Interpretation: Towards Disambiguation of Structurally Ambiguous Spoken Utterances in Indonesian
Ruhiyah Widiaputri, Sakriani Sakti (Japan Advanced Institute of Science and Technology)
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextAudio
🎯 What it does: This paper constructs a speech recognition and meaning interpretation system for Indonesian structural ambiguous sentences. First, 400 ambiguous sentences (totaling 4,800 audio recordings) were collected and annotated. Subsequently, through two frameworks—Cascade (ASR+TD) and Direct (SD)—utilizing acoustic features such as Mel-spectrogram, F0, and energy, along with meaning labels, the system automatically disambiguates ambiguous sentences into unambiguous text.
🎯 What it does: Construct a dictionary memory using synthetic speech, and during inference, combine KNN matching with a trie structure to perform dictionary-level zero-shot adaptation on existing Transducer and Whisper ASR models, thereby improving the recognition accuracy of rare words.
CodeRetrievalRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Developed a structured prediction-based entity linking system called SPEL, which accurately maps text fragments to Wikipedia entities through subword-level classification and aggregation strategies.
SPT: Learning to Selectively Insert Prompts for Better Prompt Tuning
Wei Zhu (East China Normal University), Ming Tan (Southern University of Science and Technology)
CodeClassificationComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Learning to automatically select which intermediate layers in a pre-trained model (PTM) to insert instance-aware prompts, thereby improving prompt tuning performance.
Statistical Depth for Ranking and Characterizing Transformer-Based Text Embeddings
Parker Seegmiller (Dartmouth College), Sarah Masud Preum (Dartmouth College)
CodeClassificationExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a Transformer-based Text Embedding Statistical Depth (TTE Depth) method to perform center-outlier ranking on text collections, and designs a Wilcoxon rank-sum test based on this depth to determine whether the distributions of two groups of text embeddings are significantly different.
StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models
Sullam Jeoung (University of Illinois at Urbana-Champaign), Jana Diesner (University of Illinois at Urbana-Champaign)
CodeExplainability and InterpretabilityLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Built a framework called STEREOMAP based on the psychology Stereotype Content Model (SCM) to quantify the stereotypes of large language models (LLMs) regarding the warmth and competence dimensions of social groups.
STINMatch: Semi-Supervised Semantic-Topological Iteration Network for Financial Risk Detection via News Label Diffusion
Xurui Li, Xiaozhong Liu (Worcester Polytechnic Institute)
CodeData-Centric LearningGraph Neural NetworkTransformerTextGraphBenchmarkFinance Related
🎯 What it does: Propose a semi-supervised semantic-topology iterative network, STINMatch, integrated with the news-enterprise knowledge graph (NEKG), to achieve financial risk detection.
StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding
Cheng Jiayang (Hong Kong University of Science and Technology), Zheng Zhang (Westlake University)
CodeRecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningContrastive LearningTextBenchmark
🎯 What it does: Built a large-scale story-level analogy corpus named STORYANALOGY, and designed annotation and evaluation methods for entity similarity and relation similarity based on an extended structural mapping theory. Subsequently, systematically evaluated the model's performance on story analogy recognition and generation tasks.
StructGPT: A General Framework for Large Language Model to Reason over Structured Data
Jinhao Jiang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
CodeTransformerLarge Language ModelPrompt EngineeringGraphTabular
🎯 What it does: Designed a unified framework called StructGPT, which constructs specialized interfaces (KG, tables, databases) to enable large language models (LLMs) to first read relevant structured data and then perform reasoning, thereby enhancing the zero-shot/few-shot performance of LLMs on structured data reasoning tasks.
SuperDialseg: A Large-scale Dataset for Supervised Dialogue Segmentation
Junfeng Jiang (University of Tokyo), Akiko Aizawa (National Institute of Informatics)
CodeSegmentationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed a large-scale supervised dialogue segmentation dataset called SuperDialseg and conducted benchmark experiments on multiple models.
SUT: Active Defects Probing for Transcompiler Models
Mengnan Qi (Microsoft Cloud and AI), Neel Sundaresan (Microsoft Cloud and AI)
CodeExplainability and InterpretabilityData-Centric LearningAI Code AssistantLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Developed an evaluation framework based on syntax unit testing called SUT, which includes interpretable SUT Acc and SETS metrics, capable of accurately diagnosing syntax errors in cross-language translation models.
Syntactic Substitutability as Unsupervised Dependency Syntax
Jasper Jian (Stanford University), Siva Reddy (Mila Quebec AI Institute and McGill University)
CodeTransformerLarge Language ModelText
🎯 What it does: Propose an unsupervised dependency syntactic parsing method called SSUD based on sentence syntactic substitutability, leveraging the averaging of BERT's self-attention distributions to extract syntactic information;
🎯 What it does: Propose a collaborative multi-task framework called TacoPrompt based on prompt learning, which uses a self-supervised classifier to accomplish tasks, capable of simultaneously identifying appropriate parent-child node pairs for new concepts.
🎯 What it does: To address gender bias in multilingual machine translation, we propose a target-language-agnostic gender-aware contrastive learning method called GACL, which corrects gender bias by injecting contextual gender information into encoder representations.
Target-oriented Proactive Dialogue Systems with Personalization: Problem Formulation and Dataset Curation
Jian Wang (Hong Kong Polytechnic University), Wenjie Li (Hong Kong Polytechnic University)
CodeData SynthesisTransformerLarge Language ModelAgentic AIText
🎯 What it does: Studied a personalized goal-oriented active dialogue system, and automatically constructed the TOPDIAL dataset using a role-playing large model.
Task-Agnostic Low-Rank Adapters for Unseen English Dialects
Zedian Xiao (Stanford University), Diyi Yang (Stanford University)
CodeDomain AdaptationComputational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose the HyperLoRA method, leveraging expert linguistic knowledge and hypernetworks to generate low-rank LoRA adapters, achieving task-agnostic dialect adaptation.
🎯 What it does: Designed and implemented a stance detection model called TATA that integrates theme-related (TAW) and theme-unrelated (TAG) embeddings, pre-trained using unlabeled news data and synthetic data, ultimately achieving text and topic stance classification in zero-shot and few-shot scenarios.
Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning
Dong-Ho Lee (University of Southern California), Jay Pujara (University of Southern California)
CodeTransformerLarge Language ModelPrompt EngineeringGraph
🎯 What it does: Achieving temporal knowledge graph (TKG) prediction through in-context learning (ICL) on large language models (LLM) without requiring additional training or explicit knowledge graph structure.
Text encoders bottleneck compositionality in contrastive vision-language models
Amita Kamath (University of California, Los Angeles), Kai-Wei Chang (University of California, Los Angeles)
CodeRepresentation LearningTransformerLarge Language ModelVision Language ModelAuto EncoderContrastive LearningTextMultimodalityBenchmark
🎯 What it does: By constructing a progressively increasing set of compositional statements, CompPrompts, and corresponding ControlledImCaps image-text pairs, we investigate the expression bottleneck in the single-vector text encoder of vision-language models (e.g., CLIP), and propose to assess the information loss using only a text recovery probe.
Text Rendering Strategies for Pixel Language Models
Jonas Lotz (University of Copenhagen), Desmond Elliott (Johns Hopkins University)
CodeComputational EfficiencyRepresentation LearningTransformerVision Language ModelAuto EncoderImageText
🎯 What it does: Investigated the impact of different text rendering strategies on model performance in pixel language models (PIXEL) and proposed an improved method based on character bigrams (BIGRAMS) rendering.
The ACL OCL Corpus: Advancing Open Science in Computational Linguistics
Shaurya Rohatgi (Pennsylvania State University), Min-Yen Kan (National University of Singapore)
CodeClassificationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextGraph
🎯 What it does: Constructed the ACL OCL corpus, collecting and organizing structured full texts, metadata, charts, and links to the Semantic Scholar knowledge graph from 73,285 papers in the ACL Anthology spanning 1952 to 2022.
CodeTransformerLarge Language ModelPrompt EngineeringVision Language ModelTextMultimodality
🎯 What it does: Designed a 'Socratic questioning' recursive prompting algorithm, enabling LLMs to decompose and solve complex reasoning tasks through self-asking and self-answering.
The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models
Xinyi Chen (University of Amsterdam), Sandro Pezzelle (University of Amsterdam)
CodeRepresentation LearningTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelTextMultimodalityBenchmark
🎯 What it does: This paper constructs a new multimodal benchmark dataset called BLA to evaluate the ability of pre-trained vision-language models in handling basic language structures (active-passive voice, coordination, relative clauses). Systematic experiments were conducted on models such as CLIP, ViLBERT, LXMERT, BLIP2, and OpenFlamingo under different settings, including zero-shot, fine-tuning, and context learning.
The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining
Ting-Rui Chiang (University of Southern California), Dani Yogatama (University of Southern California)
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelTextSequential
🎯 What it does: Investigate whether the distributional assumptions of masked language model pre-training can explain its sample efficiency and generalization advantages in downstream tasks.
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
Lovisa Hagström (Chalmers University of Technology), Richard Johansson (Chalmers University of Technology)
CodeRetrievalTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper evaluates the factual consistency of large language models, improves the ParaRel benchmark to ParaRel*, and systematically analyzes the sources and enhancement methods of consistency through scaling and retrieval augmentation experiments on the LLaMA and Atlas models.
The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis
Pranav Narayanan Venkit (Pennsylvania State University), Shomir Wilson (Pennsylvania State University)
CodeSafty and PrivacyTextReview/Survey Paper
🎯 What it does: Conduct a comprehensive survey of 189 peer-reviewed papers, analyzing the current state of sentiment analysis (SA) in terms of definitions, applications, models, datasets, and biases, while proposing an ethical checklist and improvement suggestions.
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
Siru Ouyang (University Of Illinois Urbana Champaign), Jiawei Han (University Of Illinois Urbana Champaign)
CodeTransformerTextChain-of-Thought
🎯 What it does: Perform self-annotation and clustering on the large-scale user-GPT interaction data from ShareGPT, comparing the task domains with those of traditional NLP benchmarks.
🎯 What it does: This paper proposes a Token-Level Masking (TLM) training strategy, which randomly masks tokens in the Transformer self-attention layer. By forcing the model to rely on neighbor information during training to generate robust representations, it achieves a regularization effect.
TOD-Flow: Modeling the Structure of Task-Oriented Dialogues
Sungryull Sohn (LG AI Research), Honglak Lee (LG AI Research)
CodeGenerationGraph Neural NetworkText
🎯 What it does: This paper proposes the TOD-Flow graph model, which automatically learns task structures from dialog data with dialog behavior annotations and uses them as conditions for any dialog policy or end-to-end generation model to improve dialog behavior prediction and response quality.
ToViLaG: Your Visual-Language Generative Model is Also An Evildoer
Xinpeng Wang (Tongji University), Xing Xie (Microsoft Research Asia)
CodeGenerationSafty and PrivacyTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper systematically studies the potential generation of toxic content (including insults, violence, pornography, etc.) by visual-language generation models (VLGMs) and proposes corresponding evaluation and detoxification methods.
🎯 What it does: Built and trained a named entity recognition model based on Airbnb listings to reveal the relationship between the use of place names in NYC Airbnb descriptions and urban spatial patterns (such as gentrification).
Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performance
Shaomu Tan (University of Amsterdam), Christof Monz (University of Amsterdam)
CodeGenerationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Systematically evaluated and quantified performance differences of zero-shot neural machine translation (NMT) across 40 languages and 1,560 translation directions, analyzing the main factors causing these differences.
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
Yifan Hou (ETH Zürich), Mrinmaya Sachan (EPFL)
CodeExplainability and InterpretabilityTransformerText
🎯 What it does: By designing a Mechanistic Probe to recover reasoning trees from the model's attention patterns in multi-step reasoning tasks, this study investigates whether language models truly perform multi-step reasoning.
Towards Example-Based NMT with Multi-Levenshtein Transformers
Maxime Bouthors (Sorbonne Université), François Yvon (Sorbonne Université)
CodeGenerationExplainability and InterpretabilityKnowledge DistillationData-Centric LearningTransformerTextRetrieval-Augmented Generation
🎯 What it does: Propose Multi-Levenshtein Transformer (TM-N-LevT), which can simultaneously edit and merge multiple retrieved fuzzy matching sentences, followed by iterative refinement to improve the accuracy and interpretability of machine translation.
Yujie Feng (Hong Kong Polytechnic University), Xiao-Ming Wu (Hong Kong Polytechnic University)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper first systematically evaluates the performance of ChatGPT on multi-domain dialog state tracking (DST) tasks, and proposes an LLM-driven DST framework LDST based on small open-source models (e.g., LLaMA), achieving performance comparable to or even better than ChatGPT through assembly domain-slot instruction tuning and parameter-efficient fine-tuning.
Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models
Weishi Wang (Salesforce AI Research), Shafiq Joty (Salesforce AI Research)
CodeMeta LearningAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies low-resource automatic program repair (APR) and proposes the Meta-APR framework, which achieves few-shot error repair by leveraging meta-learning and the pre-trained code model CodeT5.
trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback
Alexander Havrilla (CarperAI), Louis Castricato (vectorshift.ai)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposed and implemented the open-source framework trlX for fine-tuning large language models with over 70B parameters using reinforcement learning from human feedback (RLHF), supporting online algorithms (PPO, A2C) and offline algorithms (ILQL), and compatible with various parallel training strategies;
Uncertainty Guided Global Memory Improves Multi-Hop Question Answering
Alsu Sagirova (Moscow Institute of Physics and Technology), Mikhail Burtsev (London Institute for Mathematical Sciences)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose an uncertainty-guided global explicit memory (GEMFormer) to improve multi-hop question answering (MHQA), by aggregating low-entropy (high-confidence) document tokens into a memory sequence, which is then combined with local context and input into a Transformer for reasoning.
Understanding the Role of Input Token Characters in Language Models: How Does Information Loss Affect Performance?
Ahmed Alajrami (University of Sheffield), Nikolaos Aletras (University of Sheffield)
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper explores the impact of information loss on model performance by restricting the input of pre-trained language models to subsets of single-character, two-character, or three-character components (including initial characters, final characters, middle characters, vowels, or consonants) of each word, and conducts fine-tuning and evaluation on GLUE/SuperGLUE tasks and six probing tasks.
UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning
Ahmed Masry (York University), Shafiq Joty (Nanyang Technological University)
CodeGenerationKnowledge DistillationTransformerVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Propose UniChart, an end-to-end chart pre-training model that combines an image encoder (Donut + Swin Transformer) and a text decoder (BART), and learns the visual, textual, and numerical reasoning capabilities of charts through multi-task pre-training.
CodeRetrievalComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Propose UPRISE, a lightweight retrieval-based prompter that enhances the zero-shot performance of LLMs by training a retriever on a small LLM to retrieve prompts from a pre-built pool that are applicable to any zero-shot task.
Using Artificial French Data to Understand the Emergence of Gender Bias in Transformer Language Models
Lina Conti (Fondazione Bruno Kessler), Guillaume Wisniewski (Université Paris Cité)
CodeExplainability and InterpretabilityData-Centric LearningTransformerText
🎯 What it does: Train a Transformer language model using a manually created French corpus based on PCFG to investigate how it learns gender information and exhibits gender bias.
VECHR: A Dataset for Explainable and Robust Classification of Vulnerability Type in the European Court of Human Rights
Shanshan Xu, Matthias Grabmair (Graduate Institute of International and Development Studies)
CodeClassificationExplainability and InterpretabilityTransformerLarge Language ModelTextBenchmark
🎯 What it does: Constructed and released the VECHR dataset for identifying and explaining types of vulnerability in cases from the European Court of Human Rights.
🎯 What it does: This paper addresses hate speech detection in low-resource languages, proposing and evaluating three data augmentation methods based on Vicinal Risk Minimization (SSMBA, MIXUP, MIXAG), and conducting few-shot cross-lingual transfer experiments on the multi-domain, multi-language XHATE-999 dataset.
Hassan Shahmohammadi (University of Tübingen), Hendrik Lensch
CodeImage TranslationGenerationRetrievalKnowledge DistillationTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: Train a lightweight language model called ViPE to generate visual descriptions from any text, thereby helping text-to-image models better present metaphors and non-literal expressions.
Visually-Situated Natural Language Understanding with Contrastive Reading Model and Frozen Large Language Models
Geewook Kim (NAVER Cloud AI), Seunghyun Park (NAVER Cloud AI)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Proposes the Contrastive Reading Model (Cream), a multimodal framework integrating a visual encoder, auxiliary encoder, and contrastive learning to enhance understanding of text-rich images and achieve soft visual prompting fusion with large language models (LLMs).
CodeGenerationLarge Language ModelVision Language ModelMultimodality
🎯 What it does: Propose a framework named VLIS, which combines visual language models (VLM) with vision-free text language models during the inference phase to enhance language understanding and visual alignment capabilities in multimodal text generation.
We Are What We Repeatedly Do: Inducing and Deploying Habitual Schemas in Persona-Based Responses
Benjamin Kane (University of Rochester), Lenhart Schubert (University of Rochester)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes generating 'habitual event patterns (schema)' from simple self-knowledge facts, and using retrieved patterns to guide large language models (LLM) in generating dialogue responses consistent with character personas.
🎯 What it does: Propose a voting filtering mechanism based on program execution results to remove spurious programs in weakly supervised semantic parsing.
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining
Emanuele Bugliarello (Google DeepMind), Lisa Hendricks (University of Copenhagen)
CodeRetrievalRepresentation LearningVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose two weakly supervised pre-training methods based on scene graphs (Verbalised Scene Graphs and Masked Relation Classification), leveraging a small amount of manually annotated visual relationships to enhance the fine-grained visual-linguistic understanding capability of multi-modal pre-training models.
Well Begun is Half Done: Generator-agnostic Knowledge Pre-Selection for Knowledge-Grounded Dialogue
Lang Qin (Nankai University), Zhenglu Yang (Nankai University)
CodeGenerationRetrievalGraph Neural NetworkTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextGraphRetrieval-Augmented Generation
🎯 What it does: This paper proposes GATE, a model-agnostic knowledge pre-selection method that first unifies text and knowledge graphs into a graph structure, uses a graph attention network to score nodes, and adaptively selects varying amounts of knowledge through reinforcement learning, providing high-quality context for subsequent dialogue generation.
What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability
Mario Giulianelli (University of Amsterdam), Barbara Plank (LMU Munich)
CodeGenerationExplainability and InterpretabilityTransformerText
🎯 What it does: This paper evaluates whether the uncertainty of neural text generators across different tasks matches human-generated variability by performing instance-level multi-dimensional distance analysis (lexical, syntactic, semantic) on multi-reference datasets for four natural language generation tasks.
When Language Models Fall in Love: Animacy Processing in Transformer Language Models
Michael Hanna (University of Amsterdam), Sandro Pezzelle (University of Amsterdam)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This study treats pre-trained Transformer language models (GPT-2, OPT, LLaMA) as subjects in psycholinguistic experiments, investigating their behaviors in typical and atypical animacy processing, primarily by comparing the models' prediction probabilities (surprisal) with human EEG N400 responses.
When Reviewers Lock Horns: Finding Disagreements in Scientific Peer Reviews
Sandeep Kumar (Indian Institute of Technology Patna), Asif Ekbal (Indian Institute of Technology Patna)
CodeClassificationTransformerLarge Language ModelText
🎯 What it does: This paper proposes the task of automatically identifying contradictions in peer review comments and constructs a large dataset of conflicting review comment pairs named ContraSciView.
WSDMS: Debunk Fake News via Weakly Supervised Detection of Misinforming Sentences with Contextualized Social Wisdom
Ruichao Yang (Hong Kong Baptist University), Zhiwei Yang (Hong Kong Baptist University)
CodeClassificationGraph Neural NetworkTextGraph
🎯 What it does: This paper proposes a weakly supervised multi-instance learning framework called WSDMS, which utilizes social media conversation trees to detect misleading sentences in news articles and infer the overall truthfulness of the article under the condition of having only article authenticity labels.
You Told Me That Joke Twice: A Systematic Investigation of Transferability and Robustness of Humor Detection Models
Alexander Baranov (HSE University), Pavel Braslavski (HSE University)
CodeClassificationAdversarial AttackData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Systematically trained and evaluated humor detection models based on RoBERTa, Naïve Bayes, and large language models, testing their generalization and robustness through cross-validation, adversarial attacks, and supplementary datasets.
🎯 What it does: Studied how to utilize multi-source language adapters (LA) for zero-shot cross-lingual transfer without labeled target language data. Proposed an architecture ZGUL that fuses multi-source LA during training and further fine-tunes attention weights via entropy minimization at test time.