ACL 2023 Papers with AI Summaries
Annual Meeting of the Association for Computational Linguistics · 1074 papers
→ ACL 2023 papers with code (412)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 1074 ACL 2023 papers by keyword, author or institution —
free trial on arXivSub.
(QA)^2: Question Answering with Questionable Assumptions
Najoung Kim (Boston University), Jackson Petty (New York University)
Data-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Proposed a public-domain question-answering (QA) dataset (QA2) specifically for evaluating the robustness of QA systems when facing questions containing suspicious hypotheses (i.e., hypotheses that are false or unverifiable), and conducted end-to-end evaluations of multiple models on the subtasks of suspicious hypothesis detection and verification within this dataset.
A Better Way to Do Masked Language Model Scoring
Carina Kauf (Massachusetts Institute of Technology), Anna A. Ivanova (Massachusetts Institute of Technology)
Explainability and InterpretabilityTransformerLarge Language ModelScore-based ModelText
🎯 What it does: This paper addresses the sentence scoring problem in masked language models (MLM) by proposing an improved pseudo log-likelihood (PLL) metric—PLL-word-l2r—which can more accurately evaluate sentence probabilities.
A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models
Alessandro Stolfo (ETH Zürich), Mrinmaya Sachan (ETH Zürich)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper constructs a causal graph and proposes a quantitative evaluation framework for the robustness of mathematical reasoning language models. By applying controlled interventions on inputs (such as operands and textual surface forms), it calculates the Total Causal Effect (TCE) and Direct Causal Effect (DCE) to assess the model's sensitivity and robustness under different interventions.
A Cautious Generalization Goes a Long Way: Learning Morphophonological Rules
Salam Khalifa (Stony Brook University), Owen Rambow (Stony Brook University)
GenerationText
🎯 What it does: Propose a rule-based morphological generation method called PARLA, which can automatically learn Arabic morpho-phonological rules from extremely low-resource corpora.
A Close Look into the Calibration of Pre-trained Language Models
Yangyi Chen (University of Illinois at Urbana-Champaign), Heng Ji (University of Illinois at Urbana-Champaign)
Explainability and InterpretabilityTransformerText
🎯 What it does: Studied the calibration of pre-trained language models (PLM), systematically analyzed the dynamic changes in calibration during training, and evaluated existing non-learning and learnable calibration methods.
A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairment
Jiyue Jiang (University of Hong Kong), Chuan Wu (University of Hong Kong)
GenerationTransformerLarge Language ModelTextAlzheimer's DiseaseRetrieval-Augmented Generation
🎯 What it does: Constructed the Chinese Cognitive Stimulation (CS) dialogue dataset CSConv, and proposed a multi-source knowledge fusion CS dialogue system (CSD), achieving emotion support and treatment principle integrated dialogue for elderly cognitive rehabilitation;
A Comparative Study on the Impact of Model Compression Techniques on Fairness in Language Models
Krithika Ramesh (Microsoft Research), Sunayana Sitaram (Microsoft Research)
CompressionExplainability and InterpretabilityKnowledge DistillationTransformerTextBenchmark
🎯 What it does: Evaluate the impact of three compression techniques—pruning, distillation, and quantization—on the fairness of monolingual and multilingual language models, and construct a comprehensive benchmark experiment.
A Compare-and-contrast Multistage Pipeline for Uncovering Financial Signals in Financial Reports
Jia-Huei Ju, Chuan-Ju Wang (Research Center for Information Technology Innovation Academia Sinica)
ClassificationRecognitionDomain AdaptationTransformerLarge Language ModelTextFinance Related
🎯 What it does: This paper proposes a multi-stage pipeline based on year comparison to identify key financial signals in annual financial reports;
A Critical Evaluation of Evaluations for Long-form Question Answering
Fangyuan Xu (University of Texas at Austin), Eunsol Choi (University of Texas at Austin)
TransformerLarge Language ModelTextReview/Survey Paper
🎯 What it does: This paper conducts a systematic study on evaluation methods for long-form question answering (LFQA), including expert-level human evaluation and reasoning generation, as well as assessing the correlation between various automatically generated metrics and human preferences.
A Cross-Modality Context Fusion and Semantic Refinement Network for Emotion Recognition in Conversation
Xiaoheng Zhang (Beihang University), Yang Li (Beihang University)
RecognitionGraph Neural NetworkTransformerMultimodality
🎯 What it does: Propose a cross-modal context fusion and semantic refinement network (CMCF-SRNet) for emotion recognition in dialogues.
A Crosslingual Investigation of Conceptualization in 1335 Languages
Yihong Liu (LMU Munich), Hinrich Schütze (LMU Munich)
Computational EfficiencyRepresentation LearningText
🎯 What it does: Investigate conceptual differences across 1335 languages, proposing the Conceptualizer method to align concepts across languages on parallel corpora and construct bidirectional bipartite graphs
A Dataset of Argumentative Dialogues on Scientific Papers
Federico Ruggeri (University of Bologna), Iryna Gurevych (Technical University of Darmstadt)
Data SynthesisTransformerLarge Language ModelTextBenchmark
🎯 What it does: Created the ArgSciChat dataset, consisting of 41 two-person dialogues between scientists discussing 20 NLP papers, covering exploratory and argumentative intents.
A Diverse Set of Freely Available Linguistic Resources for Turkish
Duygu Altinok (Deepgram)
ClassificationRecognitionTransformerSupervised Fine-TuningText
🎯 What it does: This study constructs a free resource for Turkish NLP, including various corpora (named entity recognition, sentiment analysis, health-related comments, COVID-19 symptoms, etc.), the first Turkish pre-trained model using spaCy (containing lemmatization, morphological analysis, POS, dependency parsing, NER, etc.), as well as a series of video and code tutorials to help users quickly get started and train models.
A dynamic programming algorithm for span-based nested named-entity recognition in O(n^2)
Caio Corro (Universite Paris-Saclay)
RecognitionComputational EfficiencyRecurrent Neural NetworkTransformerText
🎯 What it does: Proposed a span-based nested named entity recognition algorithm with quadratic time complexity
A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations
Wenjie Zheng (Nanjing University of Science and Technology), Shijin Wang (iFLYTEK AI Research)
RecognitionRepresentation LearningConvolutional Neural NetworkTransformerVideoMultimodality
🎯 What it does: Designed a two-stage framework called FacialMMT, first extracting accurate facial sequences of real speakers through multimodal face recognition, unsupervised clustering, and face matching, then enhancing multimodal emotion recognition with frame-level expression recognition.
A Fast Algorithm for Computing Prefix Probabilities
Franz Nowak (ETH Zurich), Ryan Cotterell (ETH Zurich)
OptimizationComputational Efficiency
🎯 What it does: This paper proposes an improved dynamic programming algorithm for efficiently computing all prefix probabilities in probabilistic context-free grammars (PCFG) in CNF form.
A fine-grained comparison of pragmatic language understanding in humans and language models
Jennifer Hu (Massachusetts Institute of Technology), Edward Gibson (Massachusetts Institute of Technology)
ClassificationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Conduct a fine-grained comparison of seven rhetorical phenomena (lies, indirect speech, irony, conversational maxims, metaphor, humor, coherence), using zero-shot prompting to evaluate the performance of multiple language models and humans on these tasks, and analyze error patterns and linguistic cues.
A Gradient Control Method for Backdoor Attacks on Parameter-Efficient Tuning
Naibin Gu (Chinese Academy of Sciences), Weiping Wang (Chinese Academy of Sciences)
Adversarial AttackTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper proposes a gradient control method targeting backdoor attacks in parameter-efficient fine-tuning (PET) scenarios to reduce backdoor forgetting during user fine-tuning processes.
A Holistic Approach to Reference-Free Evaluation of Machine Translation
Hanming Wu (Beijing Jiaotong University), Jinan Xu (Beijing Jiaotong University)
GenerationKnowledge DistillationData-Centric LearningTransformerContrastive LearningText
🎯 What it does: Proposes a complete no-reference machine translation evaluation method called ReFreeEval, comprehensively considering fluency, word-level faithfulness, and sentence-level faithfulness;
A Length-Extrapolatable Transformer
Yutao Sun (Tsinghua University), Furu Wei (Microsoft)
TransformerLarge Language ModelTextSequential
🎯 What it does: Propose a Transformer model (LEX Transformer) capable of training on short texts and reasoning on long texts, achieving length extrapolation through improved position encoding and attention masks.
A Measure-Theoretic Characterization of Tight Language Models
Li Du (Johns Hopkins University), Ryan Cotterell (ETH Zürich)
GenerationRecurrent Neural NetworkTransformer
🎯 What it does: This paper conducts a measure-theoretic analysis of the probability distribution in autoregressive language models, defining and proving the 'compactness' property, and clarifying how probability mass leaks into infinitely long sequences.
A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spaces
Gabriella Chronis (University of Texas at Austin), Katrin Erk (University of Texas at Austin)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Automatically analyze the semantic construction of vocabulary in different grammatical structures using large language models and an interpretable psycholinguistic feature space.
A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues
Yunxin Li (Harbin Institute of Technology), Min Zhang (Meituan)
Representation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Proposed a multimodal context reasoning framework called ModCR, specifically designed for conditional reasoning based on common clues from given text premises and images;
A Natural Bias for Language Generation Models
Clara Meister (ETH Zürich), Adhiguna Kuncoro (DeepMind)
GenerationTransformerText
🎯 What it does: In the final linear layer of Transformer language generation models, initializing the bias as the log-unigram distribution of the training corpus provides the model with a natural prior, improving learning efficiency and performance.
A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization
Lining Zhang (New York University), João Sedoc (New York University)
Data-Centric LearningLarge Language ModelText
🎯 What it does: Designed and implemented a three-stage MTurk recruitment pipeline based on qualification, endurance, and reference evaluation to select workers capable of producing highly consistent summary evaluations.
A Neural Divide-and-Conquer Reasoning Framework for Image Retrieval from Linguistically Complex Text
Yunxin Li, Min Zhang (Harbin Institute of Technology)
RetrievalTransformerVision Language ModelImageTextMultimodality
🎯 What it does: Proposes an end-to-end neural divide-and-conquer reasoning framework called NDCR to address the problem of retrieving images from complex language descriptions.
A New Aligned Simple German Corpus
Vanessa Toborek (University of Bonn), Pascal Welke (University of Bonn)
Data-Centric LearningTransformerLarge Language ModelContrastive LearningTextBenchmark
🎯 What it does: Proposed a parallel corpus containing 708 articles and 10,304 sentence-aligned pairs to support German text simplification tasks.
A New Dataset and Empirical Study for Sentence Simplification in Chinese
Shiping Yang (Peking University), Xiaojun Wan (Peking University)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper creates the Chinese Sentence Simplification Evaluation dataset CSS, conducts data analysis, experiments with unsupervised, zero/few-shot methods, and evaluates large language models;
A New Direction in Stance Detection: Target-Stance Extraction in the Wild
Yingjie Li (University of Illinois at Chicago), Cornelia Caragea (University of Illinois at Chicago)
ClassificationRecurrent Neural NetworkTransformerSupervised Fine-TuningText
🎯 What it does: Proposed a task of simultaneously extracting targets and stances from social media text, along with a two-phase implementation framework.
A Novel Table-to-Graph Generation Approach for Document-Level Joint Entity and Relation Extraction
Ruoyu Zhang (Wangxuan Institute of Computer Technology, Peking University), Lei Zou (TopGraph.AI)
RecognitionGenerationGraph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: Proposed an end-to-end model TAG based on table filling to graph generation, jointly performing document-level entity recognition and relation extraction to address the error propagation problem in traditional pipeline models.
A Probabilistic Framework for Discovering New Intents
Yunhua Zhou (Fudan University), Xipeng Qiu (Fudan University)
ClassificationRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: Propose a probabilistic framework that optimizes intent allocation using the EM algorithm, leveraging prior label knowledge and contrastive learning to explore the internal structure of unlabeled data during the process.
A Simple and Effective Framework for Strict Zero-Shot Hierarchical Classification
Rohan Bhambhoria (Queen's University), Xiaodan Zhu (Rakuten Institute of Technology)
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a method that redefines hierarchical classification as a long-tail prediction task by integrating large language models (LLMs) with entailment-contradiction predictors to achieve strict zero-shot classification.
A Simple and Flexible Modeling for Mental Disorder Detection by Learning from Clinical Questionnaires
Hoyun Song (Korea Advanced Institute of Science and Technology), Jong Park
ClassificationTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Construct and evaluate a Multi-Head Siamese Network (MHS) that directly learns and detects mental disorders from text using DSM-5 symptom descriptions.
A Simple Concatenation can Effectively Improve Speech Translation
Linlin Zhang (Alibaba Group), Luo Si (Alibaba Group)
Knowledge DistillationTransformerTextMultimodalityAudio
🎯 What it does: Designed and implemented a unified cross-modal concatenation speech translation framework (uccST), which concatenates speech with transcribed text for joint encoding, and enhances performance through multi-task learning and regularization.
A Study on the Efficiency and Generalization of Light Hybrid Retrievers
Man Luo (Arizona State University), Peyman Heidari (Meta Reality Lab)
RetrievalComputational EfficiencyKnowledge DistillationContrastive LearningText
🎯 What it does: Proposed a lightweight hybrid retriever (Hybrid-LITE), further reducing the index memory of DrBoost through a LITE model trained with knowledge distillation and contrastive learning.
A Survey for Efficient Open Domain Question Answering
Qin Zhang (Shenzhen University), Meng Fang (Shenzhen University)
RetrievalCompressionComputational EfficiencyKnowledge DistillationTextReview/Survey PaperBenchmark
🎯 What it does: This paper reviews efficiency research in open-domain question answering (ODQA), categorizing three major frameworks: Retriever-Reader, Retriever-Only, and Generator-Only. It analyzes their trade-offs between memory usage, query speed, and answer quality, and provides quantitative comparisons and summaries of core technologies.
A Survey of Deep Learning for Mathematical Reasoning
Pan Lu, Kai-Wei Chang (Ucla)
TransformerLarge Language ModelPrompt EngineeringTextMultimodalityReview/Survey PaperBenchmark
🎯 What it does: This paper reviews the main tasks, datasets, models, and methods of deep learning in mathematical reasoning over the past decade, and proposes systematic classification and future research directions.
A Survey on Asking Clarification Questions Datasets in Conversational Systems
Hossein A. Rahmani (University College London), Aldo Lipani (University College London)
TransformerTextReview/Survey PaperBenchmark
🎯 What it does: Provide a comprehensive review of research on clarification questions (ACQ) in conversational systems, systematically organize and compare 77 existing papers and their used public datasets, conduct systematic analysis of evaluation metrics, experimental setups, and model performance, and provide visualization methods and benchmark experiment code.
A Survey on Zero Pronoun Translation
Longyue Wang (Tencent AI Lab), Zhaopeng Tu (Tencent AI Lab)
Contrastive LearningTextReview/Survey Paper
🎯 What it does: This paper reviews the research progress in Zero Pronoun Translation (ZPT), systematically organizing the task evolution, datasets, methods, and evaluation metrics, and proposes key findings and future directions of existing studies.
A Synthetic Data Generation Framework for Grounded Dialogues
Jianzhu Bao (Harbin Institute of Technology), Ruifeng Xu (Huawei Noah's Ark Lab)
Data SynthesisTransformerLarge Language ModelText
🎯 What it does: By constructing dialogue flows from knowledge and using large-scale pre-trained models to progressively generate dialogues, automatically synthesize grounded dialogue data for training.
A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training
Nitay Calderon (Technion - IIT), Amir Kantor (Microsoft)
GenerationKnowledge DistillationText
🎯 What it does: Systematically study the application of knowledge distillation in natural language generation tasks, propose diverse pseudo-target generation and joint teaching methods, and achieve high compression rates and performance improvements in real-world scenarios with limited annotations.
A Textual Dataset for Situated Proactive Response Selection
Naoki Otani (Megagon Labs), Eduard Hovy (University of Melbourne)
Data SynthesisRetrievalRepresentation LearningTransformerTextBenchmark
🎯 What it does: Constructed and released the SUGAR dataset for the context-aware proactive response selection task in single-turn help-seeking dialogues.
A Theory of Unsupervised Speech Recognition
Liming Wang (University of Illinois Urbana Champaign), Chang Yoo
RecognitionGenerative Adversarial NetworkAudio
🎯 What it does: This paper proposes a theoretical framework to analyze and prove the learnability, sample complexity, and GAN training dynamics of unsupervised speech recognition (ASR-U).
A Universal Discriminator for Zero-Shot Generalization
Haike Xu (Tsinghua University), Zhilin Yang (Tsinghua University)
ClassificationGenerationTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextBenchmark
🎯 What it does: Trained and evaluated a Universal Discriminator (UD) to determine whether text comes from a real distribution under zero-shot settings, unifying multiple discriminative NLP tasks under this problem.
A Weakly Supervised Classifier and Dataset of White Supremacist Language
Michael Miller Yoder (Carnegie Mellon University), Kathleen M. Carley (Carnegie Mellon University)
ClassificationTransformerLarge Language ModelText
🎯 What it does: Built a weakly supervised framework for detecting white supremacist language, using DistilBERT to perform binary classification on large-scale white supremacist corpora and neutral/anti-racist corpora;
Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations
Wenting Zhao (Cornell University), Alexander Rush (Cornell University)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Designed a self-supervised abductive commonsense reasoning method called LiPoR that does not require explainability annotations, enforcing explanation mutual exclusivity through posterior regularization to enhance the model's discriminative ability on candidate explanation sets.
Abstractive Summarizers are Excellent Extractive Summarizers
Daniel Varab (Novo Nordisk IT University of Copenhagen), Yumo Xu (University of Edinburgh)
GenerationTransformerLarge Language ModelText
🎯 What it does: This paper proposes using a pre-trained generative summarization model (e.g., BART) to directly perform extractive summarization during inference, designing three novel inference algorithms—generative ranking, generative re-ranking, and generative search—to explore the feasibility and advantages of generative models in extractive summarization.
Accelerating Transformer Inference for Translation via Parallel Decoding
Andrea Santilli (Sapienza University of Rome), Emanuele Rodola
Computational EfficiencyTransformerText
🎯 What it does: Proposes three parallel decoding algorithms (PJ, PGJ, HGJ) that accelerate Transformer machine translation inference via fixed-point iteration without retraining or modifying the model.
ACCENT: An Automatic Event Commonsense Evaluation Metric for Open-Domain Dialogue Systems
Sarik Ghazarian (University of Southern California), Nanyun Peng (University of California, Los Angeles)
TransformerSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Propose ACCENT, a reference-free automatic evaluation metric based on event-relation tuples, to measure the event commonsense proficiency of open-domain dialogue systems.
ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER
Sreyan Ghosh (University of Maryland), Dinesh Manocha (University of Maryland)
RecognitionData SynthesisTransformerLarge Language ModelText
🎯 What it does: Propose ACLM, a generative data augmentation framework based on attention-guided selective masking and text reconstruction, specifically designed for low-resource complex named entity recognition (NER).
ACTC: Active Threshold Calibration for Cold-Start Knowledge Graph Completion
Anastasiia Sedova (University of Vienna), Benjamin Roth (University of Vienna)
OptimizationRepresentation LearningData-Centric LearningGraph
🎯 What it does: Proposes the ACTC method for cold start threshold calibration in knowledge graph completion (KGC), which achieves threshold optimization for each relation through active sample selection and automatic label generation with minimal human annotations (only 10 samples).
Actively Supervised Clustering for Open Relation Extraction
Jun Zhao (Fudan University), Mingming Sun (Baidu Research)
Representation LearningData-Centric LearningTransformerAuto EncoderContrastive LearningTextBenchmark
🎯 What it does: Proposed an active supervised clustering method for open relation extraction (ASCORE), which dynamically discovers unknown relations by alternating between clustering learning and relation annotation, achieving tasks without requiring a pre-specified number of clusters;
AD-KD: Attribution-Driven Knowledge Distillation for Language Model Compression
Siyue Wu (Sun Yat-sen University), Rui Wang (Vipshop China Co Ltd)
CompressionKnowledge DistillationTransformerText
🎯 What it does: Propose an attention-based attribution knowledge distillation framework (AD-KD), which extracts the Integrated Gradients (IG) attribution distribution from the teacher model and transfers it to the student model, enabling the learning of token importance at the input level.
Adaptive and Personalized Exercise Generation for Online Language Learning
Peng Cui (ETH Zürich), Mrinmaya Sachan (ETH Zürich)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextSequential
🎯 What it does: This paper proposes a model that combines Knowledge Tracing (KT) with controllable text generation to generate adaptive and personalized language learning exercise sentences.
Advancing Multi-Criteria Chinese Word Segmentation Through Criterion Classification and Denoising
Tzu Hsuan Chou, Hung-Yu Kao (National Cheng Kung University)
ClassificationSegmentationTransformerLarge Language ModelText
🎯 What it does: Propose a multi-criteria Chinese word segmentation model based on input prompts, achieving model simplification and automatic criterion selection through criterion classification and criterion denoising objectives.
Aggregating Multiple Heuristic Signals as Supervision for Unsupervised Automated Essay Scoring
Cong Wang (Nanjing University), Qing Gu (Nanjing University)
TransformerLarge Language ModelText
🎯 What it does: Proposes the ULRA framework, which trains neural automatic essay scoring models under unsupervised conditions by aggregating multiple heuristic quality signals.
ALERT: Adapt Language Models to Reasoning Tasks
Ping Yu (Meta AI), Asli Celikyilmaz (Meta AI)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes ALERT, a new benchmark that systematically evaluates the performance of large language models on 10 fine-grained reasoning skills (such as logic, causality, common sense, text entailment, mathematics, induction, analogy, inference, spatial reasoning, and argumentation), and uses this benchmark to deeply investigate the impact of reasoning ability during three stages: pre-training, regular fine-tuning, and chain-of-thought (CoT) fine-tuning.
AlignScore: Evaluating Factual Consistency with A Unified Alignment Function
Yuheng Zha (UC San Diego), Zhiting Hu (UC San Diego)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose ALIGNSCORE, a novel fact consistency evaluation metric, which uses a unified text alignment function to determine the factual alignment between generated text and context.
Alleviating Over-smoothing for Unsupervised Sentence Representation
Nuo Chen (Hong Kong University of Science and Technology), Daxin Jiang
Representation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose a Self-Contrastive Learning (SSCL) method, which uses the hidden representations of intermediate layers from pre-trained language models (PLMs) as negative samples in unsupervised sentence representation learning, to alleviate the over-smoothing problem and improve the quality of sentence representations.
Ambiguous Learning from Retrieval: Towards Zero-shot Semantic Parsing
Shan Wu (Chinese Information Processing Laboratory), Le Sun (Chinese Information Processing Laboratory)
RetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Propose a zero-shot semantic parsing framework called Retrieval as Ambiguous Supervision (RaAS), which retrieves multiple candidate semantic representations via a pre-trained language model and treats them as ambiguous supervision. Subsequently, a confidence-driven self-training method is used to iteratively expand the candidate set and re-estimate confidence, thereby enhancing the performance of the semantic parser.
AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model
I-Hung Hsu (University of Southern California), Nanyun Peng (University of California, Los Angeles)
GenerationTransformerPrompt EngineeringText
🎯 What it does: Proposes AMPERE, a generative event argument extraction model that injects abstract meaning representations (AMR) through hierarchical prefix injection, and introduces an adjusted copying mechanism.
AMR-based Network for Aspect-based Sentiment Analysis
Fukun Ma (Tsinghua University), Lijie Wen (Tsinghua University)
ClassificationRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelText
🎯 What it does: Proposed a Path Aggregation Relation Network (APARN) based on Abstract Meaning Representation (AMR) for Fine-grained Sentiment Analysis (ABSA)
AMRs Assemble! Learning to Ensemble with Autoregressive Models for AMR Parsing
Abelardo Carlos Martínez Lorenzo (Babelscape), Roberto Navigli (Sapienza University of Rome)
Computational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningAuto EncoderTextGraphBenchmark
🎯 What it does: This paper studies integrated methods for AMR parsing, proposes a fusion model based on Transformer and a selection strategy based on perplexity, and enhances parsing quality through structural constraint detection algorithms.
An (unhelpful) guide to selecting the best ASR architecture for your under-resourced language
Robert Jimerson, Emily Prud’hommeaux
RecognitionTransformerSupervised Fine-TuningReview/Survey PaperAudio
🎯 What it does: Evaluated the performance of four mainstream ASR architectures across eleven low-resource languages
An AMR-based Link Prediction Approach for Document-level Event Argument Extraction
Yuqing Yang (Fudan University), Zheng Zhang (Amazon)
Graph Neural NetworkTransformerTextGraph
🎯 What it does: Reformulate the document-level event argument extraction problem as link prediction on a customized AMR graph, directly leveraging the graph structure to identify relationships between triggers and arguments.
An Embarrassingly Easy but Strong Baseline for Nested Named Entity Recognition
Hang Yan (Fudan University), Xipeng Qiu (Fudan University)
RecognitionConvolutional Neural NetworkTransformerBiomedical Data
🎯 What it does: This study proposes stacking a CNN on the score matrix of span-based NER models to capture spatial correlations between neighboring spans, thereby enhancing performance in nested named entity recognition.
An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models
Zhongbin Xie (University of Oxford), Thomas Lukasiewicz (Vienna University of Technology)
TransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Evaluation of parameter-efficient debiasing methods for pre-trained language models.
An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation
Xuancheng Huang (Meituan), Yang Liu (Tsinghua University)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed the Prompt Gating method, achieving a scalable plug-and-play mechanism for multi-dimensional controllable text generation through trainable gate plugins
An Inclusive Notion of Text
Ilia Kuznetsov (Technical University of Darmstadt), Iryna Gurevych (Technical University of Darmstadt)
Explainability and InterpretabilityData-Centric LearningText
🎯 What it does: This paper studies text concepts in natural language processing, proposing a unified terminology and a two-layer classification framework to describe language and non-language information in text as well as contextual relationships, aiming to enhance the reproducibility and explainability of research.
An Inner Table Retriever for Robust Table Question Answering
Weizhe Lin (University Of Cambridge), Gonzalo Iglesias (Amazon Alexa Ai)
RetrievalTransformerTabularRetrieval-Augmented Generation
🎯 What it does: This paper proposes an Internal Table Retriever (ITR), which addresses the problem of information loss caused by truncating long tables in table question answering by selecting the most relevant rows and columns through dense retrieval technology and combining them into a sub-table.
An Invariant Learning Characterization of Controlled Text Generation
Carolina Zheng (Columbia University), David Blei (Columbia University)
GenerationDomain AdaptationTransformerSupervised Fine-TuningText
🎯 What it does: Frame controlled text generation as an out-of-distribution generalization problem, model it as Invariant Learning, and explore how environment partitioning can enhance toxicity detection in filtering-based generation.
An Open Dataset and Model for Language Identification
Laurie Burchell (University of Edinburgh), Kenneth Heafield (University of Edinburgh)
ClassificationData-Centric LearningText
🎯 What it does: Created a manually reviewed monolingual corpus covering 201 languages with a total of 121 million lines, and trained a fastText language identification model on this dataset.
An Ordinal Latent Variable Model of Conflict Intensity
Niklas Stoehr (ETH Zuerich), Aaron Schein (University of Chicago)
TabularTime Series
🎯 What it does: Proposed an ordinal latent variable-based conflict intensity measurement model, inferring using four dimensions of events: subject, predicate, quantification, and object.
Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast
Yiduo Guo (Peking University), Nan Duan (Microsoft)
Domain AdaptationRepresentation LearningTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Analyze and reduce the cross-lingual performance gap in multilingual models during single-source language fine-tuning, and propose a four-strategy algorithm combining slow and fast fine-tuning.
Analyzing Text Representations by Measuring Task Alignment
Cesar Gonzalez-Gutierrez (Universitat Politècnica de Catalunya), Ariadna Quattoni (Universitat Politècnica de Catalunya)
ClassificationRepresentation LearningTransformerText
🎯 What it does: The paper proposes Task Hierarchy Alignment Score (THAS), which evaluates the alignment between text representations and classification tasks through hierarchical clustering and label probability assessment, and verifies the correlation between this score and few-shot learning performance.
Analyzing Transformers in Embedding Space
Guy Dar (Tel Aviv University), Jonathan Berant (Tel Aviv University)
Explainability and InterpretabilityRepresentation LearningTransformerText
🎯 What it does: This paper proposes a zero-channel method for interpreting Transformer parameters in the embedding space, which projects all weights (keys, values, attention matrices, etc.) using the embedding matrix and analyzes their semantics and functions in this space;
Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarization
Rongxin Zhu (University of Melbourne), Jey Han Lau (University of Melbourne)
ClassificationAnomaly DetectionData-Centric LearningTransformerTextBenchmark
🎯 What it does: Create a fine-grained sentence-level fact error dataset DIASUMFACT, evaluate and compare existing fact error detection models, and propose an unsupervised error detection method ENDERANKER.
Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution
Nupoor Gandhi (Carnegie Mellon University), Emma Strubell (Carnegie Mellon University)
Domain AdaptationTextBiomedical Data
🎯 What it does: This study proposes a method that uses only mention annotations rather than complete coreference chain annotations, to achieve efficient transfer of coreference models to new domains;
Annotation-Inspired Implicit Discourse Relation Classification with Auxiliary Discourse Connective Generation
Wei Liu (Heidelberg Institute for Theoretical Studies gGmbH), Michael Strube (Heidelberg Institute for Theoretical Studies gGmbH)
ClassificationTransformerLarge Language ModelText
🎯 What it does: Propose an end-to-end neural model that first generates implicit textual connectors between texts, then uses the generated connectors along with arguments to predict implicit discourse relations.
Answering Ambiguous Questions via Iterative Prompting
Weiwei Sun (Shandong University), Zhaochun Ren (Shandong University)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposes the AmbigPrompt model, which achieves multi-answer generation for ambiguous questions through iterative prompting (retrospective prompting), employing an alternating workflow between the prompt model and the answer generation model;
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning
Soumya Sanyal (University of Southern California), Xiang Ren (University of Southern California)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: By filtering sentences from Wikipedia based on logical reasoning keywords and using a self-supervised selective masked language model with sentence classification loss, the pre-trained language model is continuously pre-trained to enhance its logical reasoning ability.
Are Experts Needed? On Human Evaluation of Counselling Reflection Generation
Zixiu Wu (Philips Research), Daniele Riboni (University of Cagliari)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: In this study, the authors investigated whether laypeople can replace experts in assessing the coherence and contextual consistency of psychotherapy reflections by conducting binary coherence evaluations of reflections generated by GPT-2 and GPT-3, as well as genuine human reflections, with 9 non-experts and 9 professional counselors.
Are Fairy Tales Fair? Analyzing Gender Bias in Temporal Narrative Event Chains of Children’s Fairy Tales
Paulina Toro Isaza (IBM Research), Dakuo Wang (Northeastern University)
Data-Centric LearningText
🎯 What it does: This study constructs an automated text processing pipeline capable of extracting main characters, gender attributes, event chains, and thematic roles (subjects/objects) of characters within events from children's fairy tales, further performing event type annotation and bias analysis.
Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations
Bingsheng Yao (Rensselaer Polytechnic Institute), Dakuo Wang (Northeastern University)
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper designs an objective metric to evaluate the quality of human natural language explanations and proposes a unified prompt format that maps different tasks into a multiple-choice generation task. Subsequently, fine-tuning and inference experiments were conducted on five public datasets using T5 and BART models to verify the effectiveness of the metric.
Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-text Rationales
Brihi Joshi (University of Southern California), Xiang Ren (University of Southern California)
Explainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Investigate the extent to which free-text rationales generated by large language models assist humans in solving tasks, propose a human practicality evaluation framework, and introduce an automated metric, GEN‑U, to assess and enhance the usefulness of model-generated rationales for humans.
Are Message Passing Neural Networks Really Helpful for Knowledge Graph Completion?
Juanhui Li (Michigan State University), Dawei Yin (Baidu Inc)
Representation LearningGraph Neural NetworkContrastive LearningGraphBenchmark
🎯 What it does: This study systematically evaluates the practicality of message passing neural networks (MPNN) in knowledge graph completion (KGC), first replacing the message passing layer in MPNN with a simple multi-layer perceptron (MLP), then comparing its performance with the original MPNN; subsequently, it deeply explores the impact of scoring functions and loss functions (negative sampling strategies) on model performance, and constructs a simple ensemble method based on MLP, ultimately achieving performance comparable to or even surpassing traditional MPNN.
Are Pre-trained Language Models Useful for Model Ensemble in Chinese Grammatical Error Correction?
Chenming Tang (Peking University), Yunfang Wu (Peking University)
TransformerLarge Language ModelText
🎯 What it does: Conduct model integration experiments on Chinese grammar error correction tasks, attempting to select the best output using perplexity calculated by pre-trained language models.
Are Sample-Efficient NLP Models More Robust?
Nelson F. Liu (Stanford University), Robin Jia (University of Southern California)
TransformerSupervised Fine-TuningPrompt EngineeringTextBiomedical Data
🎯 What it does: Investigate the relationship between sample efficiency and model robustness, systematically evaluating the performance of three modeling interventions (prompt tuning, model scale enhancement, pre-training data expansion) across 14 different datasets
Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark
Wenjun Peng (University of Science and Technology of China), Xing Xie (Microsoft Research Asia)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: Proposed an EmbMarker method based on backdoors to implant inheritable watermarks in Embedding-as-a-Service (EaaS), aiming to protect the copyrights of large language models (LLMs).
ArgAnalysis35K : A large-scale dataset for Argument Quality Analysis
Omkar Joshi (COEP Technological University), Yashodhara Haribhakta (COEP Technological University)
Data-Centric LearningRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Built and released a dataset named ArgAnalysis35K containing 35,000 argument-analysis pairs, covering multi-topic arguments and corresponding analytical explanations from debaters at different skill levels;
ArgU: A Controllable Factual Argument Generator
Sougata Saha (State University of New York at Buffalo), Rohini Srihari (State University of New York at Buffalo)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed ArgU, a controllable factual argument generator capable of generating diverse argument texts based on given stances and Walton argument schemes, and released the largest corpus with 69,428 annotated arguments including argument schemes and fact spans.
Attention as a Guide for Simultaneous Speech Translation
Sara Papi (Fondazione Bruno Kessler), Marco Turchi (Independent Researcher)
GenerationComputational EfficiencyTransformerTextAudio
🎯 What it does: Proposed a novel adaptive decision strategy called EDATT, which determines when to output partial translations or wait for more audio input by observing encoder-decoder attention weights in offline-trained speech translation models.
AtTGen: Attribute Tree Generation for Real-World Attribute Joint Extraction
Yanzeng Li (Peking University), Lei Zou (Peking University)
GenerationConvolutional Neural NetworkRecurrent Neural NetworkText
🎯 What it does: Unify the attribute extraction task as a tree generation problem, and propose AtTGen, a text-to-tree generation model, to achieve joint attribute extraction in closed-domain, open-domain, and semi-open-domain scenarios.
Attractive Storyteller: Stylized Visual Storytelling with Unpaired Text
Dingyi Yang (Renmin University of China), Qin Jin (Renmin University of China)
GenerationTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposes the stylized visual story generation task (SVST) for unpaired text, and designs a multi-task memory-enhanced framework called StyleVSG. It jointly trains on real image sequences and unpaired style text to generate coherent stories with specific writing styles.
Attributable and Scalable Opinion Summarization
Tom Hosking (University of Edinburgh), Mirella Lapata (University of Edinburgh)
GenerationExplainability and InterpretabilityTransformerAuto EncoderText
🎯 What it does: Propose an unsupervised opinion summarization method that encodes review sentences into a hierarchical discrete latent space, identifies common opinions based on the frequency of latent codes, and generates interpretable abstract and extractive summaries based on these frequent subpaths.
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Zichun Yu (Tsinghua University), Zhiyuan Liu (Tsinghua University)
RetrievalDomain AdaptationTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Proposed a general-purpose retrieval plugin called Augmentation-Adapted Retriever (AAR), which enhances the zero-shot generalization of large language models by training the retriever using attention signals from a small source model without fine-tuning the target language model.
AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models
Siheng Li (Tsinghua University), Yujiu Yang (Tsinghua University)
GenerationData SynthesisKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Fine-tune large language models (LLMs) using a small amount of human dialogue, transforming information-seeking dialogue generation tasks into language modeling tasks to generate synthetic dialogues, and distill LLM knowledge to small task models through two-stage training.
Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluations
Lucy Lu Wang (University of Washington), Byron C. Wallace (Northeastern University)
GenerationTransformerTextBiomedical DataBenchmark
🎯 What it does: This paper proposes a human evaluation dataset to assess the quality of medical multi-document summarization (MDS) and analyzes the gap between existing automatic evaluation metrics and human assessments.
Automatic Annotation of Direct Speech in Written French Narratives
Noé Durandard (Deezer Research), Elena Epure
ClassificationRecognitionRecurrent Neural NetworkTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Automatic annotation of direct speech in French novel texts and the construction of a unified evaluation framework.
Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations
Hyunjae Kim (Korea University), Jaewoo Kang (Korea University)
RecognitionData SynthesisTransformerTextBiomedical DataBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the HighGEN framework, which combines sentence retrieval and phrase embedding retrieval to automatically construct a pseudo-dictionary with high coverage, and generates a NER dataset using a weakly supervised approach.