ACL 2023 Papers — Page 6
Annual Meeting of the Association for Computational Linguistics · 1074 papers
In-Context Analogical Reasoning with Pre-Trained Language Models
Xiaoyang Hu (University of Michigan), Joyce Chai (University of Michigan)
Representation LearningTransformerLarge Language ModelPrompt EngineeringImageTextBenchmarkChain-of-Thought
🎯 What it does: Leverage pre-trained language models (e.g., OPT, GPT-3) in a zero-shot setting to convert perceptual features of the visual analogy task Raven Progressive Matrices (RPM) into language abstraction (naming and decomposition), then use text prompts to enable the model to infer missing matrix items.
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation
Qi Jia (Shanghai Jiao Tong University), Kenny Zhu
GenerationTransformerText
🎯 What it does: Proposed and implemented In-sample Curriculum Learning (ICL), which achieves an easy-to-difficult training order by first letting the model generate the last few words of a sentence, then gradually reducing the prefix length until full sequence generation.
Incorporating Attribution Importance for Improving Faithfulness Metrics
Zhixue Zhao (University of Sheffield), Nikolaos Aletras (University of Sheffield)
ClassificationExplainability and InterpretabilityTransformerText
🎯 What it does: This paper proposes using a 'soft erase' technique to randomly mask input word vectors in proportion to their importance, thereby improving traditional sparse hard erase (comprehensiveness, sufficiency) evaluation methods when assessing the reliability of feature attribution (FA) methods.
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization
Dongqi Liu (Saarland University), Vera Demberg (Saarland University)
GenerationTransformerLarge Language ModelText
🎯 What it does: Propose RSTformer, which improves long text summarization by leveraging relation types and uncertainty information from Rhetorical Structure Theory (RST)
Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions
John Chung, Saleema Amershi (Microsoft Research)
ClassificationData SynthesisData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Generate diverse and accurate text classification training data by combining large language models (LLM) with human intervention
IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation Metrics for Indian Languages
Ananya Sai B (Indian Institute of Technology Madras), Raj Dabre (National Institute of Information and Communications Technology)
TransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed a fine-grained multidimensional quality assessment (MQM) annotated dataset with 7,000 items to evaluate the quality of machine translation from English to five Indian languages (Hindi, Marathi, Tamil, Malayalam, and Gujarati).
InfoMetIC: An Informative Metric for Reference-free Image Caption Evaluation
Anwen Hu (Renmin University of China), Qin Jin (Renmin University of China)
Explainability and InterpretabilityRepresentation LearningTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: Proposed a no-reference image caption evaluation metric called InfoMetIC, which can identify fine-grained erroneous words and undescribed regions, and provide coarse-grained metrics including text precision, visual recall, and overall quality scores.
Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling
Shengqiong Wu (National University of Singapore), Tat-Seng Chua (National University of Singapore)
ClassificationRepresentation LearningGraph Neural NetworkTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: This study addresses multi-modal relation extraction (MRE) by proposing a framework that simultaneously achieves internal information shielding and external information utilization. The framework first represents images and text with visual scene graphs (VSG) and text scene graphs (TSG), respectively, then fuses them into a cross-modal graph (CMG). Subsequently, the CMG is subjected to structure fine-grained screening via graph information bottleneck (GIB) to remove irrelevant noise. Finally, a latent multi-modal topic model (LAMO) is introduced to provide external semantic context for relation extraction, ultimately completing relation prediction.
infoVerse: A Universal Framework for Dataset Characterization with Multidimensional Meta-information
Jaehyung Kim (KAIST), Dongyeop Kang (University of Minnesota)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed a general framework named infoVerse that leverages multiple model-driven meta-information to perform multi-dimensional representation of datasets, and designed a subset selection method based on Determinantal Point Process (DPP) in this space.
Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognition
Haodong Zhao (Tianjin University), Jing Xu (Tianjin University)
ClassificationTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose a parameter-efficient multi-layer implicit discourse relation recognition framework (PEMI), achieving multi-layer relation classification with a minimal number of trainable parameters.
Injecting knowledge into language generation: a case study in auto-charting after-visit care instructions from medical dialogue
Maksim Eremeev (New York University), Anitha Kannan (Curai Health)
GenerationTransformerTextBiomedical Data
🎯 What it does: This study explores how to generate post-visit care instructions in medical dialogues, with a focus on the accuracy of rare words during the generation process.
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
Wenhao Zhu (Nanjing University), Jiajun Chen (University of Hong Kong)
GenerationTransformerSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Propose a training framework called INK, which cyclically injects k-NN knowledge during training using a small number of adapter parameters, gradually smoothing the representation space of neural machine translation models, and removing dependence on large-scale retrieval libraries during inference.
Instruction Induction: From Few Examples to Natural Language Task Descriptions
Or Honovich (Tel Aviv University), Omer Levy (Tel Aviv University)
Data-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper studies the ability of large language models to explicitly generate natural language task instructions (Instruction Induction) given only a few input-output examples.
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Harsh Trivedi (Stony Brook University), Ashish Sabharwal (Allen Institute for AI)
RetrievalTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes the IRCoT method, which alternates between Chain-of-Thought (CoT) reasoning and retrieval to achieve a mutually guiding cycle between retrieval and reasoning in knowledge-intensive multi-step question answering;
Interpretable Math Word Problem Solution Generation via Step-by-step Planning
Mengxue Zhang (University of Massachusetts Amherst), Andrew Lan (University of Massachusetts Amherst)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposed a planning-based step generation method that uses a language model to generate intermediate solution steps for math word problems in a step-by-step manner and provides interpretable problem-solving processes.
Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis
Mario Giulianelli (University of Amsterdam), Andrey Kutuzov (University of Oslo)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes utilizing automatically generated contextualized word definitions as interpretable word representations, applying them to semantic change analysis, and further constructing a visual semantic dynamic graph by building semantic labels from the generated definitions, helping researchers explain the trajectory of semantic evolution over time.
Interpreting Positional Information in Perspective of Word Order
Zhang Xilong (Xidian University), Liang Xuefeng (Xidian University)
Explainability and InterpretabilityRepresentation LearningTransformerText
🎯 What it does: By introducing a weight concatenation operation, the permutation invariance of the Transformer attention module is broken, indirectly achieving positional information encoding for word order; further formalizing it as a positional kernel and designing PosNet (at the attention-level and embedding-level) to realize implicit positional information injection.
Introducing Semantics into Speech Encoders
Derek Xu (University of California, Los Angeles), Wei Wang (Meta AI)
Representation LearningTransformerLarge Language ModelGenerative Adversarial NetworkAudio
🎯 What it does: Inject semantic information into unsupervised speech encoders to improve performance on SLU and SQA tasks.
Is Anisotropy Truly Harmful? A Case Study on Text Clustering
Mira Ait-Saada (Université Paris Cité), Mohamed Nadif (Université Paris Cité)
Representation LearningTransformerText
🎯 What it does: This paper post-processes the text representations of pre-trained models such as BERT and RoBERTa, and evaluates their representational capacity through document clustering tasks, exploring the impact of spatial anisotropy on clustering performance.
Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detection
Rheeya Uppaal (University of Wisconsin-Madison), Yixuan Li (University of Wisconsin-Madison)
Anomaly DetectionTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: Propose and evaluate a distance-based zero-shot OOD detection method without any fine-tuning of pre-trained language models, and compare it with various fine-tuning objectives (CE, TAPT, SupCon) and output layer baseline methods (MSP, energy).
Is GPT-3 a Good Data Annotator?
Bosheng Ding (Nanyang Technological University), Lidong Bing (Alibaba Group)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Evaluate the feasibility and effectiveness of GPT-3 as a data annotation tool across various NLP tasks;
Joint Constrained Learning with Boundary-adjusting for Emotion-Cause Pair Extraction
Huawen Feng (South China University of Technology), Qianli Ma (South China University of Technology)
RecognitionGraph Neural NetworkTransformerContrastive LearningText
🎯 What it does: Proposed the Joint Constrained Learning and Boundary-adjusting framework to address the data imbalance problem in Emotional-Cause Pair Extraction (ECPE).
Joint Document-Level Event Extraction via Token-Token Bidirectional Event Completed Graph
Qizhi Wan (Stony Brook University), Rong Hu (Jiangxi University of Finance and Economics)
Recurrent Neural NetworkGraph Neural NetworkTransformerTextFinance Related
🎯 What it does: Proposes a joint document-level event extraction method based on a Token-Token bidirectional event completion graph, and implements the corresponding edge prediction model EDEE.
Joint End-to-end Semantic Proto-role Labeling
Elizabeth Spaulding (University of Colorado Boulder), Mark Dredze (Bloomberg Lp)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Built an end-to-end joint model that can simultaneously identify predicates and arguments and perform semantic proto-role annotation on arguments.
Jointprop: Joint Semi-supervised Learning for Entity and Relation Extraction with Heterogeneous Graph-based Propagation
Yandan Zheng (Nanyang Technological University), Anh Tuan Luu (Nanyang Technological University)
RecognitionRepresentation LearningGraph Neural NetworkTransformerSupervised Fine-TuningTextGraph
🎯 What it does: Propose Jointprop, a joint semi-supervised learning framework that processes named entity recognition and relation extraction tasks simultaneously by propagating labels through a heterogeneous graph.
Just Like a Human Would, Direct Access to Sarcasm Augmented with Potential Result and Reaction
Changrong Min, Hongfei Lin (Dalian University Of Technology)
ClassificationGraph Neural NetworkText
🎯 What it does: This study proposes a novel sarcasm detection method called SD-APRR, which augments sarcastic text with potential outcomes and human reactions inferred by COMET, to more comprehensively express the negative context within sarcasm.
KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understanding
Shangbin Feng (University of Washington), Yulia Tsvetkov (University of Washington)
Graph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: Integrate three-layer knowledge-aware contexts, including local text, document-level graph structure, and global knowledge graph, to propose the KALM model for long document understanding.
KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment
Lingzhi Wang (Chinese University of Hong Kong), Hongzhi Yin (University of Queensland)
Safty and PrivacyText
🎯 What it does: Proposed a general machine learning forgetting framework KGA, achieving approximate data forgetting for NLP tasks through knowledge gap alignment;
KILM: Knowledge Injection into Encoder-Decoder Language Models
Yan Xu (Amazon Alexa AI), Dilek Hakkani-Tur
GenerationRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Utilizing continuous pre-training, inject structured knowledge from Wikipedia, including entity links and their short descriptions, into the Encoder-Decoder language model BART, enabling the model to acquire entity-related knowledge without modifying the architecture or increasing parameters.
kNN-TL: k-Nearest-Neighbor Transfer Learning for Low-Resource Neural Machine Translation
Shudong Liu (University of Macau), Min Zhang (Harbin Institute of Technology)
GenerationDomain AdaptationTransformerTextRetrieval-Augmented Generation
🎯 What it does: Propose a k-Nearest Neighbor Transfer Learning (k NN-TL) framework to enable continuous utilization of the parent model's knowledge throughout the initialization, training, and inference processes in low-resource neural machine translation.
KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations
Myeongjun Jang (University of Oxford), Oana-Maria Camburu (University College London)
Explainability and InterpretabilityAdversarial AttackLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Proposed a fast consistency attack eKnowIA based on external knowledge, and introduced the KNOW method to alleviate NLE inconsistencies through knowledge injection;
Knowledge of cultural moral norms in large language models
Aida Ramezani (University of Toronto), Yang Xu (University of Toronto)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This study investigates whether English pre-trained language models (EPLMs) contain cross-cultural moral norms, and evaluates their performance across different countries through fine-grained inference using data from the World Values Survey and the PEW Global Attitudes survey;
Knowledge Transfer in Incremental Learning for Multilingual Neural Machine Translation
Kaiyu Huang (Tsinghua University), Yang Liu (Tencent)
Domain AdaptationKnowledge DistillationTransformerText
🎯 What it does: This paper proposes a knowledge transfer method for incremental learning in multilingual neural machine translation (MNMT), utilizing the embedding and feed-forward network parameters of external models as pluggable modules, while freezing the original model parameters to achieve efficient adaptation to new language pairs.
Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Joel Jang (Korea Advanced Institute of Science and Technology), Minjoon Seo (Korea Advanced Institute of Science and Technology)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Achieve knowledge forgetting by performing gradient ascent (i.e., maximizing loss) on the target token sequence of pre-trained language models, thereby eliminating extractable private information without retraining the model.
Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversations
Yang Deng, Wai Lam (The Chinese University of Hong Kong)
TransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Propose a hybrid proactive emotional support dialog system KEMI, integrating knowledge retrieval with multi-task generation to enhance system proactiveness and information quality.
Knowledgeable Parameter Efficient Tuning Network for Commonsense Question Answering
Ziwang Zhao (Beijing University of Posts and Telecommunications), Yequan Wang (Beijing Academy of Artificial Intelligence)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Propose a knowledge-aware parameter-efficient fine-tuning network called KPE, which injects external knowledge into a frozen pre-trained language model through parameter-sharing adapters to enhance commonsense question answering performance.
Label-Aware Hyperbolic Embeddings for Fine-grained Emotion Classification
Chih Yao Chen, Lun-Wei Ku (Institute of Information Science Academia Sinica)
ClassificationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a HypEmo framework that integrates hyperbolic space label embedding with RoBERTa for fine-grained emotion classification;
LAIT: Efficient Multi-Segment Encoding in Transformers with Layer-Adjustable Interaction
Jeremiah Milbauer (Google Research), Tal Schuster (Google Research)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose LAIT (Layer-Adjustable Interaction Transformer), which divides the input into semantic segments. It first performs parallel encoding within each segment and then applies unified cross-segment self-attention in subsequent layers, preserving the Transformer's context modeling capability while significantly reducing computational cost.
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language
Mehran Kazemi (Google Research), Deepak Ramachandran (Google Research)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Propose an algorithm called LAMBADA that integrates natural language reasoning with backward chaining, utilizing a language model to perform fact checking, rule selection, goal decomposition, and symbolic consistency judgment, constructing verifiable proof chains.
Language Detoxification with Attribute-Discriminative Latent Space
Jin Myung Kwak (KAIST), Sung Ju Hwang (KAIST)
GenerationSafty and PrivacyTransformerLarge Language ModelText
🎯 What it does: Text detoxification generation using a single pre-trained language model (GPT-2) in a projected discriminative latent space.
Language model acceptability judgements are not always robust to context
Koustuv Sinha (Meta AI), Adina Williams (Meta AI)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Investigating the robustness of large language models in judging acceptability under different contexts (length, structural similarity, presence of grammatical errors).
Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions
Himanshu Thakur (Carnegie Mellon University), Louis-Philippe Morency (Carnegie Mellon University)
Explainability and InterpretabilityRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Apply 'data intervention' using a small number (10 samples) of gender bias training data on pre-trained language models, followed by fine-tuning to reduce gender bias in the models.
Language of Bargaining
Mourad Heddaya (University of Chicago), Alexander Zentefis (Yale University)
TransformerTextFinance RelatedAudio
🎯 What it does: Collected and annotated house buying and selling negotiation dialogues under laboratory conditions, constructing a comparative dataset contrasting natural language communication with alternating offer communication, and performing fine-grained speech behavior annotations on negotiation semantics;
Large Language Models Are Reasoning Teachers
Namgyu Ho (KAIST), Se-Young Yun (KAIST)
Knowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
🎯 What it does: Leverage large-scale language models to generate chain-of-thought examples, followed by fine-tuning small models to achieve significant capabilities in complex reasoning tasks;
Large Language Models Meet NL2Code: A Survey
Daoguang Zan (Cooperative Innovation Center, Institute of Software, Chinese Academy of Sciences), Jian-Guang Lou (Integrative Innovation Center, Institute of Software, Chinese Academy of Sciences)
GenerationTransformerLarge Language ModelTextReview/Survey PaperBenchmark
🎯 What it does: This paper systematically reviews and summarizes 27 existing large language models (LLMs) for natural language to code (NL2Code), and evaluates their performance on benchmarks such as HumanEval and MBPP;
Large-Scale Correlation Analysis of Automated Metrics for Topic Models
Jia Peng Lim (Singapore Management University), Hady Lauw
Text
🎯 What it does: Conduct large-scale correlation analysis between automatic consistency metrics and human judgments, sampling thousands of model-free generated topics on three corpora, and designing fine-grained user studies to examine the consistency between metrics and human evaluations.
Large-scale Lifelong Learning of In-context Instructions and How to Tackle It
Jisoo Mok (Seoul National University), Sungroh Yoon (Seoul National University)
Representation LearningMeta LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose a lifelong learning framework named DYNAINST, which fine-tunes pre-trained language models task-by-task and leverages contextual instructions to continuously improve generalization performance at the instance and task levels.
Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings
Ta-Chung Chi (Carnegie Mellon University), Peter Ramadge
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: This paper studies Transformer language models without using position embeddings, finding that the variance of self-attention outputs contracts with position, implicitly encoding positional information.
Layer-wise Fusion with Modality Independence Modeling for Multi-modal Emotion Recognition
Jun Sun (Zhejiang Lab), Taihao Li (Zhejiang Lab)
RecognitionTransformerMultimodality
🎯 What it does: Propose a hierarchical fusion model LFMIM based on Transformer, emphasizing multimodal independence and using independent labels for training;
LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training for Document Understanding
Yi Tu (Ant Group), Jinyang Tang (Ant Group)
ClassificationRecognitionTransformerVision Language ModelMultimodality
🎯 What it does: Proposed a Transformer-based multimodal pre-training model called LayoutMask, which leverages local 1D position information and segment-level 2D position information. It designs two masking strategies: Whole Word Masking and Layout-Aware Masking, and introduces a new task called Masked Position Modeling to strengthen text-layout interaction.
Laziness Is a Virtue When It Comes to Compositionality in Neural Semantic Parsing
Maxwell Crouse (IBM Research), Tim Klinger (IBM Research)
GenerationTransformerLarge Language ModelTextGraph
🎯 What it does: Proposed a bottom-up recursively constructed neural semantic parser that employs lazy expansion of candidate operations, avoiding overfitting and compositional limitations caused by traditional autoregressive top-down generation.
Learning “O” Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER
Ruotian Ma (Fudan University), Yun Wen Chen
ClassificationRecognitionTransformerLarge Language ModelContrastive LearningText
🎯 What it does: To address the problem of unannotated entities in class-incremental NER, this paper proposes two methods: entity-aware contrastive learning and distance-threshold re-labeling, aiming to maintain the discriminability of old classes and enhance the learning of new classes.
Learning Action Conditions from Instructional Manuals for Instruction Understanding
Te-Lin Wu (University of California, Los Angeles), Nanyun Peng (University of California, Los Angeles)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a task of inferring preconditions and postconditions of actions from online manuals, and constructs a densely manually annotated evaluation dataset; through weakly supervised methods (keywords, entity tracking, coreference, temporal reasoning, etc.), large-scale training instances are generated, and based on this, pre-trained language models are trained in multiple stages to evaluate their performance on this task.
Learning Answer Generation using Supervision from Automatic Question Answering Evaluators
Matteo Gabburo (University of Trento), Alessandro Moschitti (Amazon Alexa AI)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: This paper investigates introducing the automatic QA evaluator GAVA as a supervisory signal during generative question answering (GenQA) training, proposing three training strategies: static data augmentation (GAVA-SDA), dynamic data augmentation (GAVA-DDA), and loss weighting (GAVA-LW) to improve the accuracy and quality of generated answers.
Learning Better Masking for Better Language Model Pre-training
Dongjie Yang (Shanghai Jiao Tong University), Hai Zhao (Shanghai Jiao Tong University)
OptimizationRepresentation LearningData-Centric LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: This paper studies and proposes two time-varying masking strategies (Masking Ratio Decay and POS-Tagging Weighted Masking) to improve the pre-training process of MLM models such as BERT.
Learning Dynamic Contextualised Word Embeddings via Template-based Temporal Adaptation
Xiaohang Tang (University of Liverpool), Danushka Bollegala (University of Liverpool)
Representation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Fine-tune pre-trained masked language models using time-sensitive templates to learn time-varying contextualized word vectors.
Learning In-context Learning for Named Entity Recognition
Jiawei Chen (Chinese Information Processing Laboratory), Le Sun (Chinese Information Processing Laboratory)
RecognitionMeta LearningTransformerLarge Language ModelText
🎯 What it does: Propose a context-based learning approach for named entity recognition, injecting this capability into pre-trained language models through meta-function pre-training, achieving the ability to recognize new entity types with only a few examples.
Learning Language-Specific Layers for Multilingual Machine Translation
Telmo Pires (Apple), Stephan Peitz (Apple)
Computational EfficiencyRepresentation LearningNeural Architecture SearchTransformerText
🎯 What it does: Propose a language-specific Transformer layer (LSL) that enhances the capacity of multilingual machine translation models for each language without increasing inference costs.
Learning Latent Relations for Temporal Knowledge Graph Reasoning
Mengqi Zhang (University of Chinese Academy of Sciences), Liang Wang (University of Chinese Academy of Sciences)
Representation LearningRecurrent Neural NetworkGraph Neural NetworkGraphTime Series
🎯 What it does: Propose a novel Temporal Knowledge Graph reasoning method called L TKG 2, which enhances entity prediction performance by constructing a structural encoder and a latent relationship learning module to embed entities and uncover implicit associations within and across time.
Learning Multi-Step Reasoning by Solving Arithmetic Tasks
Tianduo Wang (Singapore University of Technology and Design), Wei Lu (Singapore University of Technology and Design)
TransformerSupervised Fine-TuningTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes a synthetic pre-training task named MSAT, and injects multi-step reasoning capabilities into medium-sized language models (such as RoBERTa-based Seq2Seq and DAG structure models) through adapter tuning and digit-wise tokenization, thereby improving their performance on mathematical word problems (MWP).
Learning Neuro-Symbolic World Models with Conversational Proprioception
Don Joven Agravante (IBM Research), Alexander Gray (IBM Research)
Explainability and InterpretabilityReinforcement LearningWorld ModelText
🎯 What it does: Built a text game agent based on a neuro-symbolic world model, integrating semantic parsing, logic neural networks (ILP), planning, and incorporating conversational ontology-aware memory constraints.
Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback
Jing Xu (Meta AI), Jason Weston (Meta AI)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: This paper studies how internet-driven open-domain dialogue models can continuously learn new skills during the deployment phase through human feedback.
Learning Non-linguistic Skills without Sacrificing Linguistic Proficiency
Mandar Sharma (Virginia Tech), Naren Ramakrishnan (Virginia Tech)
Representation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the Skill-LM framework, achieving the injection of strict arithmetic reasoning into large language models (LLMs) without compromising language capabilities.
Learning Optimal Policy for Simultaneous Machine Translation via Binary Search
Shoutao Guo (Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences), Yang Feng (Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences)
OptimizationRecurrent Neural NetworkTransformerReinforcement LearningText
🎯 What it does: Propose a simultaneous machine translation model BS-SiMT that constructs optimal strategies online using binary search.
Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning
Subhajit Chaudhury (IBM), Alexander Gray (IBM)
Explainability and InterpretabilityRepresentation LearningTransformerReinforcement LearningText
🎯 What it does: Designed a neural-symbolic text reinforcement learning agent called NESTA, which generates symbolic triplets using AMR semantic parsing and learns abstract interpretable action rules through logical neural networks, thereby learning strategies in text games.
Learning to Generate Equitable Text in Dialogue from Biased Training Data
Anthony Sicilia (University of Pittsburgh), Malihe Alikhani (University of Pittsburgh)
GenerationSupervised Fine-TuningText
🎯 What it does: Studied methods to achieve fairness (equity) in dialogue generation, providing a formal definition of fairness (score parity). It was proven through computational learning theory that minimizing test divergence can achieve fairness, enabling fair dialogue generation without altering biased training data.
Learning to Imagine: Visually-Augmented Natural Language Generation
Tianyi Tang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
GenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelDiffusion modelTextMultimodality
🎯 What it does: Proposed the LIVE method, which achieves visual-enhanced natural language generation by dynamically generating and filtering visual images for input sentences, and then fusing them with pre-trained language models.
Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?
Chengwei Qin (Nanyang Technological University), Ruochen Zhao (Nanyang Technological University)
Meta LearningTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper proposes Meta Prompt Tuning (MPT), which learns the initialization parameters of soft prompts (prompt embeddings) via a meta-learning approach on source tasks, thereby enhancing cross-task generalization on target tasks.
Learning to Simulate Natural Language Feedback for Interactive Semantic Parsing
Hao Yan (George Mason University), Ziyu Yao (Meta AI)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a natural language feedback simulation task in interactive semantic parsing, and design a feedback evaluator and multiple feedback generators, aiming to generate high-quality feedback sentences without relying on a large amount of human-labeled data.
Learning to Substitute Spans towards Improving Compositional Generalization
Zhaoyi Li (University of Science and Technology of China), Defu Lian (University of Science and Technology of China)
Data SynthesisRepresentation LearningRecurrent Neural NetworkTransformerContrastive LearningTextBenchmark
🎯 What it does: Propose two data augmentation methods based on span replacement, SpanSub and L2S2, to enhance the compositional generalization ability of sequence models.
Learning with Partial Annotations for Event Detection
Jian Liu (Beijing Jiaotong University), Zhe Zhao (Tencent AI Lab)
Data-Centric LearningTransformerContrastive LearningText
🎯 What it does: To address the partial annotation issue in event detection, we propose a model based on trigger localization and contrastive learning, combined with a self-correction mechanism for training.
LENS: A Learnable Evaluation Metric for Text Simplification
Mounica Maddela (Georgia Institute of Technology), Wei Xu (Georgia Institute of Technology)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed a learnable text simplification evaluation metric called LENS, and constructed a large-scale human-annotated dataset named SIMPEVAL.
Let Me Check the Examples: Enhancing Demonstration Learning via Explicit Imitation
Sirui Wang (Tsinghua University), Wei Wu (Meituan Inc.)
ClassificationRepresentation LearningTransformerPrompt EngineeringContrastive LearningText
🎯 What it does: Proposes the Imitation-Demo method, which simulates the human review process by introducing contrastive learning and demonstration label re-prediction in example learning to strengthen the association between prompts and demonstrations.
Leveraging Prefix Transfer for Multi-Intent Text Revision
Ruining Chong (Beijing Language and Culture University), Erhong Yang
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose a multi-intent text revision system based on prefix-tuning, which can automatically identify and integrate multiple editing intents without requiring explicit intent annotations, generating improved text.
LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development
Ilias Chalkidis, Anders Søgaard (University Of Copenhagen)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Construct multilingual English legal corpus LeXFiles and release the LegalLAMA probing benchmark, training and evaluating two LexLM legal PLMs as well as other models' pre-training, probing, and downstream performance.
LexSym: Compositionality as Lexical Symmetry
Ekin Akyurek, Jacob Andreas (Massachusetts Institute of Technology)
Data-Centric LearningRecurrent Neural NetworkTransformerAuto EncoderImageTextMultimodality
🎯 What it does: Propose a generic data augmentation method called LEXSYM based on lexical symmetry, which leverages the homomorphism of lexical algebra to automatically discover data symmetric transformations and use them to enhance training samples.
LI-RAGE: Late Interaction Retrieval Augmented Generation with Explicit Signals for Open-Domain Table Question Answering
Weizhe Lin (University of Cambridge), Gonzalo Iglesias (Amazon Alexa AI)
GenerationRetrievalTransformerPrompt EngineeringTabularBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes the LI-RAGE framework, integrating deep table retrieval, generative answering, and explicit training signals to achieve end-to-end open-domain table question answering.
Lifting the Curse of Capacity Gap in Distilling Language Models
Chen Zhang (Beijing Institute Of Technology), Dawei Song (Beijing Institute Of Technology)
Knowledge DistillationMixture of ExpertsText
🎯 What it does: By introducing MINIMOE (Minimal Expert) in knowledge distillation, the student model's capacity is expanded without significantly increasing inference computational cost, thereby addressing the curse of capacity gap.
lilGym: Natural Language Visual Reasoning with Reinforcement Learning
Anne Wu (Cornell University), Yoav Artzi (Cornell University)
TransformerReinforcement LearningVision-Language-Action ModelImageTextMultimodalityBenchmark
🎯 What it does: Proposes lil Gym, a natural language conditioned reinforcement learning benchmark based on NLVR data, enabling agents to manipulate objects in a visual environment according to human-written natural language statements to satisfy or negate the truth value of the statements.
Limitations of Language Models in Arithmetic and Symbolic Induction
Jing Qian (University of California, Santa Barbara), Xifeng Yan (University of California, Santa Barbara)
TransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Investigate the limitations of large pre-trained language models in arithmetic and symbolic reasoning tasks, and attempt various mitigation methods.
Linear Classifier: An Often-Forgotten Baseline for Text Classification
Yu-Chen Lin (National Taiwan University), Chih-Jen Lin (National Taiwan University)
ClassificationTransformerLarge Language ModelText
🎯 What it does: Compare the performance of linear SVM and BERT in text classification, highlighting the importance of linear classifiers as essential baselines.
Linguistic representations for fewer-shot relation extraction across domains
Sireesh Gururaja (Carnegie Mellon University), Carolyn Rosé (Carnegie Mellon University)
ClassificationDomain AdaptationRepresentation LearningGraph Neural NetworkText
🎯 What it does: The study uses automatically generated dependency parsing and abstract meaning representation (AMR) graphs as additional features to enhance the model in cross-domain few-shot relation extraction tasks.
Listener Model for the PhotoBook Referential Game with CLIPScores as Implicit Reference Chain
Shih-Lun Wu (Carnegie Mellon University), Liangze Li (Carnegie Mellon University)
ClassificationTransformerVision Language ModelMultimodality
🎯 What it does: Proposed a novel listener model for PhotoBook, a collaborative dialogue game, where the model directly reads a round of dialogue and 6 images to predict whether the target image is shared with the opponent.
LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically Constructed from Live Streaming
Jingsheng Gao (Shanghai Jiao Tong University), Baoyuan Wang (Xiaobing.AI)
GenerationRetrievalTransformerLarge Language ModelVideoTextBenchmark
🎯 What it does: Constructed a Chinese personalized dialogue dataset based on live video called LiveChat, and designed and evaluated retrieval-based and generative dialogue models on it.
LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion
Dongfu Jiang (Zhejiang University), Bill Yuchen Lin (Allen Institute for Artificial Intelligence)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Developed a two-stage LLM fusion framework called LLM-BLENDER, designed to dynamically select and merge responses from multiple open-source large language models.
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning
Amirhossein Abaskohi (University of Tehran), Yadollah Yaghoobzadeh (University of Tehran)
ClassificationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningText
🎯 What it does: Propose a framework named LM-CPPF that combines prompt tuning with supervised contrastive learning. By leveraging large language models to generate synonyms under few examples, data augmentation is achieved, enhancing the model's ability to distinguish categories.
Local Byte Fusion for Neural Machine Translation
Makesh Narsimhan Sreedhar (University of Wisconsin-Madison), Junjie Hu (University of Wisconsin-Madison)
GenerationTransformerText
🎯 What it does: Proposed a byte-based local fusion method called LOBEF, which explicitly aggregates character/word-level semantics in multilingual neural machine translation using n-gram convolution and word boundary self-attention.
Local Interpretation of Transformer Based on Linear Decomposition
Sen Yang (Nanjing University), Jiajun Chen (Nanjing University)
Explainability and InterpretabilityTransformerText
🎯 What it does: Propose a method based on linear decomposition to explain ReLU-activated Transformer models, and prove that Transformers can be locally linearized when ignoring gradients of attention and layer normalization;
Log-linear Guardedness and its Implications
Shauli Ravfogel (Bar-Ilan University), Ryan Cotterell (ETH Zürich)
Safty and PrivacyTransformerText
🎯 What it does: This paper investigates whether subsequent linear classifiers can still leak attributes that have been eliminated through linear concept removal from neural network representations;
Logic-driven Indirect Supervision: An Application to Crisis Counseling
Mattia Medina Grespan (University of Utah), Vivek Srikumar (University of Utah)
ClassificationSafty and PrivacyTransformerText
🎯 What it does: Leverage low-cost session-level risk labels to achieve indirect supervision and enhancement of sentence-level risk labels in text-based crisis counseling through declarative logic constraints
Long-Tailed Question Answering in an Open World
Yi Dai (Tsinghua University), Yongbin Li (Alibaba Group)
Knowledge DistillationTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed the Open Long-Tail QA (OLTQA) task and designed a unified QA model that addresses challenges in long-tail and unseen tasks by leveraging instance-level knowledge sharing and knowledge mining from large pre-trained language models.
MAD-TSC: A Multilingual Aligned News Dataset for Target-dependent Sentiment Classification
Evan Dufraisse (Université Paris-Saclay, CEA, List), Jerome Deshayes (Université Paris-Saclay, CEA, List)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed and constructed the MAD-TSC dataset—the first multilingual aligned news target sentiment analysis (TSC) dataset, and evaluated various TSC methods based on pre-trained language models on it;
Making Language Models Better Reasoners with Step-Aware Verifier
Yifei Li (Peking University), Weizhu Chen (Microsoft Corporation)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Propose the DIVERSE method, combining diverse prompts, a voting verifier, and a step-by-step verifier to improve the accuracy of large language models on reasoning tasks.
Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation
Martijn Bartelds (University of Groningen), Martijn Wieling (University of Groningen)
RecognitionData SynthesisTransformerSupervised Fine-TuningAudio
🎯 What it does: In resource-scarce dialects/minority languages (Gronings, Frisian, Bezemer, and Nasari), transfer learning via a self-supervised pre-trained model (XLS-R) is applied, combined with data augmentation using self-training and TTS-generated synthetic speech to improve automatic speech recognition (ASR) word error rate (WER).
ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning
Xiao Xu (Harbin Institute of Technology), Nan Duan (Intel Labs)
Representation LearningTransformerMixture of ExpertsVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose the ManagerTower architecture, which introduces managers in each layer of the cross-modal encoder to aggregate insights from multi-layer pre-trained visual and text experts, achieving more comprehensive cross-modal alignment and fusion.
MANNER: A Variational Memory-Augmented Model for Cross Domain Few-Shot Named Entity Recognition
Jinyuan Fang (University of Glasgow), Yong Jiang (Alibaba Group)
RecognitionDomain AdaptationTransformerLarge Language ModelText
🎯 What it does: Propose MANNER, which leverages a memory module and optimal transport to achieve cross-domain few-shot named entity recognition;
Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
Myra Cheng (Stanford University), Dan Jurafsky (Stanford University)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Generate 'persona' descriptions for different identity groups (e.g., race, gender, or their intersections) using natural language prompts, and analyze stereotypes in these descriptions using the unsupervised, unlabeled Marked Personas framework, followed by comparison with human-written descriptions under the same prompts.
MasakhaPOS: Part-of-Speech Tagging for Typologically Diverse African languages
Cheikh M. Bamba Dione (Universite Gaston Berger), Dietrich Klakow (University Of Porto)
ClassificationTransformerTextBenchmark
🎯 What it does: Constructed and released the largest POS tag dataset for 20 African languages, MasakhaPOS, and conducted POS tagging baseline and cross-lingual transfer experiments based on this dataset.
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
Jack FitzGerald (Amazon), Prem Natarajan (Capital One)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed a multilingual natural language understanding (NLU) dataset named MASSIVE with 1 million examples, covering 51 languages, 18 domains, 60 intents, and 55 slots;
Massively Multilingual Lexical Specialization of Multilingual Transformers
Tommaso Green (University of Mannheim), Goran Glavaš (University of Mannheim)
Representation LearningTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: This paper proposes a multilingual lexical hierarchy specialization method, which uses the multilingual synonym relations from BabelNet to perform one-time alignment and fine-tuning on multilingual Transformers (mBERT, XLM-R), thereby generating superior static word vectors.