ACL 2023 Papers — Page 5
Annual Meeting of the Association for Computational Linguistics · 1074 papers
Few-shot In-context Learning on Knowledge Base Question Answering
Tianle Li (University of Waterloo), Wenhu Chen (Vector Institute)
TransformerLarge Language ModelGraphBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose a training-agnostic KBQA framework called KB-BINDER, which utilizes large language models to generate logical form drafts, then binds entities and relations on the knowledge graph, and finally executes to obtain answers.
Few-shot Reranking for Multi-hop QA via Language Model Prompting
Muhammad Khalifa (University of Michigan), Lu Wang (University of Michigan)
RetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Proposed a few-shot multi-hop retrieval re-ranking method called PROMPTRANK based on large language model prompts.
FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning
Qinyuan Ye (University of Southern California), Hannaneh Hajishirzi (Allen Institute for AI)
Computational EfficiencyMeta LearningTransformerPrompt EngineeringText
🎯 What it does: Proposed a FiD-ICL intermediate fusion method based on a fused decoder for efficient gradient-free context learning.
Finding the Pillars of Strength for Multi-Head Attention
Jinjie Ni (Nanyang Technological University), Erik Cambria (Nanyang Technological University)
Computational EfficiencyTransformerText
🎯 What it does: Design Grouped Head Attention through grouped constraint training (GCT) and voting retention (V2S) to reduce redundancy and over-parameterization in multi-head attention;
Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings
Daniel Rotem (Hebrew University of Jerusalem), Roy Schwartz (Hebrew University of Jerusalem)
ClassificationComputational EfficiencyTransformerTextBenchmark
🎯 What it does: Conduct a comparative study of adaptive inference methods (Early-Exit and Multi-Model) in low-resource scenarios, and propose an improved algorithm named SWEET to address the gradient conflict problem in Early-Exit
Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models
Zhong Zhang (University of Electronic Science and Technology of China), Junming Shao (University of Electronic Science and Technology of China)
ClassificationComputational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Investigated whether fine-tuning of pre-trained language models on downstream tasks occurs only within a very small subspace, and verified its feasibility by analyzing fine-tuning trajectories to extract task-specific low-dimensional subspaces.
FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information
Andrew Zhu (University of Pennsylvania), Chris Callison-Burch (University of Pennsylvania)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Built and utilized a large-scale real D&D game corpus named FIREBALL, providing 25k scenarios, 8M sentences, 2.1M commands, 1.2M game states, to train language models for completing two tasks: 'sentence→command' and 'state→narration'.
FLamE: Few-shot Learning from Natural Language Explanations
Yangqiaoyu Zhou (University of Chicago), Chenhao Tan (University of Chicago)
ClassificationMeta LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose a two-stage few-shot learning framework called FLamE, which first uses GPT-3 to generate natural language explanations and then fine-tunes a smaller RoBERTa model with the generated explanations for classification.
Focused Prefix Tuning for Controllable Text Generation
Congda Ma (Tokyo Institute of Technology), Manabu Okumura (Tokyo Institute of Technology)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose Focused Prefix Tuning (FPT), which reduces the negative impact of implicit attributes on controllable text generation by combining specific and general prefixes along with logits operations during inference.
Forgotten Knowledge: Examining the Citational Amnesia in NLP
Janvijay Singh (Georgia Institute of Technology), Saif Mohammad
Text
🎯 What it does: This paper systematically analyzes the citation time distribution in the field of natural language processing (NLP), exploring trends and changes in citing older papers over the past decade;
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction
Chen-Yu Lee (Google Cloud AI Research), Tomas Pfister (Google Research)
RecognitionRepresentation LearningGraph Neural NetworkTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: Proposed the FormNetV2 model, which builds upon FormNetV1 by incorporating the image modality and performing pooling on graph edges using image features;
Free Lunch for Efficient Textual Commonsense Integration in Language Models
Wanyun Cui (Shanghai University of Finance and Economics), Xingran Chen (University of Michigan)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Aggregate training samples with shared textual commonsense descriptions into the same batch to reduce the number of redundant encodings of textual commonsense;
Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averaging
Fabian David Schmidt (University of Würzburg), Goran Glavaš (University of Würzburg)
Domain AdaptationRepresentation LearningText
🎯 What it does: Proposes two simple techniques, Checkpoint Averaging (CA) and Run Averaging (RA), to enhance the robustness of zero-shot (ZS) and few-shot (FS) cross-lingual transfer (XLT), with systematic evaluation across multiple tasks.
From Characters to Words: Hierarchical Pre-trained Language Model for Open-vocabulary Language Understanding
Li Sun (Boston University), Cha Zhang (Microsoft)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: Built a hierarchical open-vocabulary language model that encodes characters into word-level representations and further contextualizes word sequences using deep Transformers;
From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models
Julia Mendelsohn (University of Michigan School of Information), Maarten Sap (Language Technologies Institute Carnegie Mellon University)
ClassificationRecognitionTransformerLarge Language ModelPrompt EngineeringTextTime Series
🎯 What it does: This study constructs a large-scale dogwhistle dictionary and proposes a new classification system, subsequently using the dictionary to perform time-series analysis on US Congressional speeches; meanwhile, it evaluates GPT-3's ability to identify and generate dogwhistles, and tests the stealthiness of dogwhistles in Perspective API's toxicity detection.
From Key Points to Key Point Hierarchy: Structured and Expressive Opinion Summarization
Arie Cattan (Bar Ilan University), Roy Bar-Haim (IBM Research)
TransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes organizing key points obtained from Key Point Analysis (KPA) hierarchically into Key Point Hierarchies (KPH), and constructs the first KPH benchmark dataset named THINKP; meanwhile, it designs and evaluates multiple methods for predicting hierarchical relationships from key points;
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models
Shangbin Feng (University of Washington), Yulia Tsvetkov (University of Washington)
Explainability and InterpretabilityTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper constructs a measurement framework based on political spectrum testing to evaluate political bias in pre-trained language models and investigates how these biases affect fairness in hate speech and misinformation detection tasks.
From the One, Judge of the Whole: Typed Entailment Graph Construction with Predicate Generation
Zhibin Chen (Peking University), Dongyan Zhao (Peking University)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: Based on seed predicates, a generative language model is used to generate new predicates, then entailment relationships between predicates are constructed to generate an expandable Typed Entailment Graph (TP-EGG).
From Ultra-Fine to Fine: Fine-tuning Ultra-Fine Entity Typing Models to Fine-grained
Hongliang Dai (Nanjing University of Aeronautics and Astronautics), Ziqian Zeng (South China University of Technology)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose a method that first trains a broad-coverage BERT model on the Ultra-Fine task, then fine-tunes it on a small number of manually labeled samples to create a Fine-Grained entity type model.
FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction
Tianshuo Peng (Wuhan University), Hai Zhao (Shanghai Jiao Tong University)
TransformerText
🎯 What it does: Proposed a generic information extraction framework FSUIE based on fuzzy boundaries, improving the overfitting issue in span prediction of traditional UIE models
FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue
Weihao Zeng (Beijing University of Posts and Telecommunications), Weiran Xu (Meituan)
Knowledge DistillationRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose the FutureTOD pre-training model, which utilizes a self-training framework to distill future dialogue information into the current dialogue representation.
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
Jian Yang (State Key Lab of Software Development Environment Beihang University), Zhoujun Li (State Key Lab of Software Development Environment Beihang University)
GenerationRepresentation LearningTransformerGenerative Adversarial NetworkText
🎯 What it does: Propose a GAN-based Encoder-Decoder pre-training framework called GANLM, which uses an auxiliary discriminator to jointly perform replacement word detection and replacement word denoising tasks, enhancing both generation and understanding capabilities.
GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decoding
Konstantin Yakovlev (Huawei Noah's Ark Lab), Irina Piontkovskaya (Huawei Noah's Ark Lab)
GenerationComputational EfficiencyTransformerLarge Language ModelAuto EncoderTextBenchmark
🎯 What it does: Proposes GEC-DePenD, a non-autoregressive grammatical error correction model that employs decoupled permutation and decoding networks to generate fully corrected text in one pass.
Generalizing Backpropagation for Gradient-Based Interpretability
Kevin Du (ETH Zürich), Ryan Cotterell (ETH Zürich)
Explainability and InterpretabilityTransformerText
🎯 What it does: Proposed a framework that generalizes backpropagation as semiring dynamic programming, enabling efficient computation of various statistics of the gradient graph (e.g., maximum gradient path, entropy).
Generating EDU Extracts for Plan-Guided Summary Re-Ranking
Griffin Adams (Columbia University), Kathleen McKeown (Columbia University)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a two-stage abstractive summarization and re-ranking method: first, use BART to generate content plans based on EDUs (Elemental Discourse Units), then use another BART model to generate summary candidates according to the plan, followed by selecting the best summary through re-rankers like BRIO; meanwhile, demonstrate how the same framework can guide GPT-3.5 to generate diverse summaries and re-rank them.
Generating Hashtags for Short-form Videos with Guided Signals
Tiezheng Yu (Hong Kong University of Science and Technology), Yi-Chia Wang (Meta AI)
GenerationRecommendation SystemTransformerVision Language ModelVideoTextMultimodalityRetrieval-Augmented GenerationAudio
🎯 What it does: Reframe the short video tag recommendation task as a generation task, proposing the Guided Generative Model (GGM). The model uses tags retrieved by a Vision-Language Model as guidance signals, combining multimodal information from video, text, and audio to generate short video tags.
Generating Structured Pseudo Labels for Noise-resistant Zero-shot Video Sentence Localization
Minghang Zheng (Peking University), Yang Liu (Peking University)
RetrievalLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Propose a zero-shot video sentence localization framework SPL, which first generates free-form pseudo queries using the BLIP image captioning model, then selects query-dependent event proposals by sliding window and intra/inter-event difference based on the similarity between the query and video frames, followed by non-maximum suppression to filter high-quality pseudo query-event pairs. Subsequently, a noise-robust training method with sample reweighting and pseudo label refinement is employed to train a fully supervised localization model.
Generating User-Engaging News Headlines
Pengshan Cai (University of Massachusetts), Dong Yu (Tencent AI Lab)
Recommendation SystemTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Built a personalized news headline generation framework based on extracting signature phrases from user reading history, and achieved unsupervised user profiling through contrastive learning;
Generating Visual Spatial Description via Holistic 3D Scene Understanding
Yu Zhao (Tianjin University), Tat-Seng Chua (National University of Singapore)
GenerationGraph Neural NetworkTransformerPrompt EngineeringVision Language ModelImagePoint Cloud
🎯 What it does: Leverage an external 3D scene extractor to obtain 3D object and scene features from images, constructing a goal-oriented 3D spatial scene graph (GO3D-SG) centered on the target object, and generating visual spatial descriptions (VSD) through this graph;
Generic Temporal Reasoning with Differential Analysis and Explanation
Yu Feng (University of Pennsylvania), Dan Roth (University of Pennsylvania)
Explainability and InterpretabilityRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextTime SeriesSequential
🎯 What it does: Propose the TODAY task and dataset, utilizing temporal difference analysis to evaluate models' temporal reasoning capabilities under subtle contextual changes, and enhancing model generalization through joint learning and interpretability supervision.
GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles
Tanmay Parekh (University of California Los Angeles), Nanyun Peng (University of California Los Angeles)
Data SynthesisPrompt EngineeringTextBenchmark
🎯 What it does: Constructed a large-scale, diverse event and argument role ontology, and generated the GENEVA benchmark dataset based on it to evaluate the generalization capability of event argument extraction (EAE) models under different resource and unknown event scenarios.
GIFT: Graph-Induced Fine-Tuning for Multi-Party Conversation Understanding
Jia-Chen Gu (University of Science and Technology of China), Guoping Hu (iFLYTEK Research)
ClassificationRecognitionGraph Neural NetworkTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose a lightweight graph-induced fine-tuning method called GIFT, which utilizes the reply graph structure of multi-party dialogues to refine attention weights within the Transformer through four types of edge relationships (reply-to, replied-by, reply-self, indirect-reply), thereby enhancing the understanding performance of multi-party dialogues.
Gloss-Free End-to-End Sign Language Translation
Kezhou Lin (Zhejiang University), Yi Yang (Zhejiang University)
Graph Neural NetworkTransformerVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: Proposed a fully gloss-annotation-free end-to-end sign language translation framework called GloFE, which leverages concept anchor words extracted from translated text for contrastive learning to supervise the joint training of the visual encoder and text decoder.
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages
Ayyoob Imani (LMU Munich), Hinrich Schütze (LMU Munich)
Representation LearningData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Trained a multilingual large model called Glot500-m covering 511 low-resource languages, and constructed the corresponding 600GB corpus Glot500-c
Going Beyond Sentence Embeddings: A Token-Level Matching Algorithm for Calculating Semantic Textual Similarity
Hongwei Wang (Tencent AI Lab), Dong Yu (Tencent AI Lab)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose a token-level matching algorithm that can be stacked onto any pre-trained language model during inference to compute semantic similarity between sentence pairs.
Gradient Ascent Post-training Enhances Language Model Generalization
Dongkeun Yoon (KAIST), Minjoon Seo (KAIST)
ClassificationGenerationData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Post-training is performed on the pre-trained OPT language models (350M, 1.3B, 2.7B) by executing a few gradient ascent steps (up to 15 steps) on random unannotated text, referred to as GAP.
Gradient-based Intra-attention Pruning on Pre-trained Language Models
Ziqing Yang, Shijin Wang (iFLYTEK Research)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a gradient-based structured pruning method called GRAIN, specifically designed for fine-grained pruning of the internal attention structure in Transformers, combined with training through knowledge distillation;
Graph-based Relation Mining for Context-free Out-of-vocabulary Word Embedding Learning
Ziran Liang (Sun Yat-sen University), Yanghui Rao (Sun Yat-sen University)
Representation LearningGraph Neural NetworkContrastive LearningText
🎯 What it does: This paper proposes a graph-based relational mining (GRM) method for out-of-vocabulary (OOV) word embedding learning without context.
GreenKGC: A Lightweight Knowledge Graph Completion Method
Yun Cheng Wang, C.-C. Jay Kuo (University of Southern California)
Computational EfficiencyRepresentation LearningGraph
🎯 What it does: Proposed a lightweight knowledge graph completion method called GreenKGC, which employs a three-stage independent training process (representation learning, feature pruning, decision learning) to achieve efficient and accurate predictions in low-dimensional spaces.
Grokking of Hierarchical Structure in Vanilla Transformers
Shikhar Murty (Stanford University), Christopher Manning (Stanford University)
TransformerText
🎯 What it does: Investigated the hierarchical generalization capability of Transformers during long-term post-training processes and found that structured 'grokking' can be achieved through extended training.
Grounded Multimodal Named Entity Recognition on Social Media
Jianfei Yu (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
RecognitionObject DetectionTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: This paper proposes a new multimodal named entity recognition task—Grounded Multimodal Named Entity Recognition (GMNER), which requires identifying entities, entity types, and corresponding visual bounding boxes from text-image social media posts; simultaneously constructs a GMNER dataset based on Twitter; and proposes a hierarchical index generation framework, H-Index, which uses BART to perform end-to-end generation of entity-type-region triplets;
Grounding Characters and Places in Narrative Text
Sandeep Soni (University of California, Berkeley), David Bamman (University of California, Berkeley)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Proposed and implemented a multi-classification task for attributing spatial relationships between characters and locations in narrative texts, constructing approximately 2500 annotated samples.
Guide the Many-to-One Assignment: Open Information Extraction via IoU-aware Optimal Transport
Kaiwen Wei (Chinese Academy of Sciences), Guo Zhi
TransformerText
🎯 What it does: Proposes an IoU-aware Optimal Transport (IOT) framework to address the one-to-many label assignment problem in Open Information Extraction (OIE) tasks, and designs an Assignment-Guided Multi-Granularity (AM) loss;
Guiding Computational Stance Detection with Expanded Stance Triangle Framework
Zhengyuan Liu (Institute for Infocomm Research Astar), Nancy Chen
ClassificationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Improved the generalization ability of stance detection by expanding the stance triangle model and performing adversarial multi-objective annotation on single-domain corpora.
HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level
Haoran Luo, Wei Lin (Beijing University of Posts and Telecommunications)
Representation LearningGraph Neural NetworkGraph
🎯 What it does: This paper proposes a hierarchical attention model HAHE for embedding hyper-relational knowledge graphs, which simultaneously considers the global hypergraph structure and local semantic sequences;
Hard Sample Aware Prompt-Tuning
Yuanjian Xu (Tsinghua University), Zaiqing Nie (Tsinghua University)
ClassificationTransformerReinforcement LearningPrompt EngineeringContrastive LearningText
🎯 What it does: Proposes the HardPT framework, which identifies hard samples through reinforcement learning and enhances prompt-tuning performance under few-shot scenarios using adaptive contrastive learning.
HAUSER: Towards Holistic and Automatic Evaluation of Simile Generation
Qianyu He (Fudan University), Yunwen Chen (DataGrand Inc)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed HAUSER——an overall automatic evaluation system for simile generation tasks, along with five evaluation metrics that cover three dimensions: quality, creativity, and informativeness.
Hearing Lips in Noise: Universal Viseme-Phoneme Mapping and Transfer for Robust Audio-Visual Speech Recognition
Yuchen Hu (Nanyang Technological University), Eng Siong Chng (Nanyang Technological University)
RecognitionDomain AdaptationRepresentation LearningTransformerMultimodality
🎯 What it does: Proposed a framework called UniVPM for audio-visual speech recognition that utilizes the visual modality for unsupervised noise adaptation.
Helping a Friend or Supporting a Cause? Disentangling Active and Passive Cosponsorship in the U.S. Congress
Giuseppe Russo (ETH Zurich), Frank Schweitzer (ETH Zurich)
ClassificationRepresentation LearningRecurrent Neural NetworkGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextGraphTabular
🎯 What it does: Study and predict the active and passive co-sponsorship behaviors of US Congress legislators, and distinguish their motivations.
HermEs: Interactive Spreadsheet Formula Prediction via Hierarchical Formulet Expansion
Wanrong He (Tsinghua University), Dongmei Zhang (Microsoft Research Asia)
AI Code AssistantTransformerLarge Language ModelTabular
🎯 What it does: Propose HERMES, a framework for predicting spreadsheet formulas through hierarchical formula expansion, and implement an interactive formula completion interface.
Hexatagging: Projective Dependency Parsing as Tagging
Afra Amini (ETH Zurich), Ryan Cotterell (ETH Zurich)
TransformerLarge Language ModelText
🎯 What it does: Propose Hexatagging, a framework that converts projective dependency parsing into a tagging task;
Hidden Schema Networks
Ramses Sanchez, Cesar Ojeda Marin
GenerationRepresentation LearningTransformerLarge Language ModelAuto EncoderTextGraph
🎯 What it does: Designed and implemented an implicit pattern network model that maps language sentences to symbolic sequences of an implicit graph via variational autoencoders, while imposing explicit relational structures on pre-trained language models.
Hierarchical Verbalizer for Few-Shot Hierarchical Text Classification
Ke Ji (Southeast University), Baoyuan Wang (Xiaobing.AI)
ClassificationTransformerPrompt EngineeringContrastive LearningText
🎯 What it does: Propose a hierarchical prompting (HierVerb) method that uses multi-layer learnable verbalizers to embed hierarchical label knowledge into pre-trained language models (PLMs) for hierarchical text classification under few-shot scenarios.
HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation
Anchun Gui (Xiamen University), Han Xiao (Xiamen University)
Computational EfficiencyTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes HiFi, a parameter-efficient fine-tuning method that only fine-tunes the important heads in multi-head attention.
HINT: Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation
Hamish Ivison (Allen Institute for AI), Matthew Peters (Allen Institute for AI)
Computational EfficiencyRepresentation LearningMeta LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a hypernetwork-based instruction tuning framework called HINT, which can map task instructions and a few examples into parameter-efficient modules (adapter and prefix) in one go, and concatenate the encoded instructions with the input during the decoding stage.
Hints on the data for language modeling of synthetic languages with transformers
Rodolfo Zevallos (Universitat Pompeu Fabra), Nuria Bel
ClassificationRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper pre-trains RoBERTa language models on five different morphological language types (English, French, German, Turkish, Quechua) at scales of 1M–6M tokens using three tokenizers (BPE, Unigram, Canonical DeepSpin), and fine-tunes them on two downstream tasks (NER and POS), systematically evaluating the impact of data size and tokenization methods on LM perplexity and downstream performance.
HiPool: Modeling Long Documents Using Graph Neural Networks
Irene Li, Rex Ying (Yale University)
ClassificationGraph Neural NetworkTransformerTextGraphBenchmark
🎯 What it does: Propose HiPool, which uses graph neural networks for hierarchical modeling of long texts to address the bottleneck in long sequence encoding.
History Semantic Graph Enhanced Conversational KBQA with Temporal Information Modeling
Hao Sun (Peking University), Yongbin Li (Alibaba Group)
Graph Neural NetworkTransformerLarge Language ModelTextGraph
🎯 What it does: Propose a dialogue-based knowledge base question answering model HSGE that utilizes historical semantic graphs and time information enhancement.
HistRED: A Historical Document-Level Relation Extraction Dataset
Soyoung Yang (KAIST), Jaegul Choo (KAIST)
TransformerTextBenchmark
🎯 What it does: Constructed HistRED—a bilingual (Korean + Chinese characters) relation extraction dataset based on historical documents from the Korean Joseon Dynasty's 'Yeonhaengrok,' and proposed a cross-lingual attention model leveraging bilingual context for document-level relation extraction.
HiTIN: Hierarchy-aware Tree Isomorphism Network for Hierarchical Text Classification
He Zhu (Beihang University), Ke Xu (Beihang University)
ClassificationGraph Neural NetworkTextBenchmark
🎯 What it does: Propose the HiTIN model, which utilizes an encoding tree constructed through structural entropy minimization in hierarchical text classification to efficiently inject label hierarchy information into text representations.
Holistic Prediction on a Time-Evolving Attributed Graph
Shohei Yamasaki (Nomura Research Institute Ltd), Makoto Onizuka (Aarhus University)
Representation LearningRecurrent Neural NetworkGraph Neural NetworkGraphTime SeriesSequential
🎯 What it does: Propose the AGATE framework, capable of comprehensively predicting the future of temporal evolving attribute graphs, including node attributes, link additions/removals, and the emergence/disappearance of new/old nodes; and implement a novel new node attribute prediction method called PROSER within the framework.
Holographic CCG Parsing
Ryosuke Yamaki (Ritsumeikan University), Daichi Mochihashi (Institute of Statistical Mathematics)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: Propose to treat Combinatory Categorial Grammar (CCG) as recursive combinations in a continuous vector space, constructing phrase- and sentence-level representations through recursive HolE embeddings; achieve high-precision super-tagging and parsing based on this; simultaneously utilize decomposability to realize phrase-level text fill-in.
How About Kind of Generating Hedges using End-to-End Neural Models?
Alafate Abulimiti (INRIA), Justine Cassell (Carnegie Mellon University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied fine-tuning pre-trained large language models (BART, DialoGPT, BlenderBot) on peer tutoring corpora, and generating context-appropriate hedge or non-hedge sentences through reranking with a hedge classifier.
How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks
Salijona Dyrmishi (University of Luxembourg), Maxime Cordy (University of Luxembourg)
Adversarial AttackText
🎯 What it does: Conduct a large-scale human evaluation on nearly 2000 texts to examine the effectiveness and naturalness of text adversarial attacks.
How Do In-Context Examples Affect Compositional Generalization?
Shengnan An (Xi'an Jiaotong University), Dongmei Zhang (Microsoft Corporation)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper investigates how the similarity, diversity, and complexity of examples in the context learning of large language models affect compositional reasoning ability, and proposes the COFE benchmark suite.
How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech
Aditya Yedetore (Boston University), R. Thomas McCoy (New York University)
GenerationRepresentation LearningRecurrent Neural NetworkTransformerText
🎯 What it does: Train LSTM and Transformer models on child-directed corpora to learn language models and question generation, examining their hierarchical inductive ability for English yes/no questions.
How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives
Xinpeng Wang (Ludwig Maximilian University Munich), Barbara Plank (Ludwig Maximilian University Munich)
CompressionKnowledge DistillationTransformerTextBenchmark
🎯 What it does: This study comprehensively evaluates the knowledge distillation targets of the BERT model, with a particular focus on the impact of weight initialization and distillation targets on model compression.
How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases
Aaron Mueller (Johns Hopkins University), Tal Linzen (New York University)
Representation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied the impact of Transformer pre-training architecture and corpus on hierarchical syntactic inductive bias.
How Well Apply Simple MLP to Incomplete Utterance Rewriting?
Jiang Li (Inner Mongolia University), Guanglai Gao (Inner Mongolia University)
ClassificationGenerationLarge Language ModelText
🎯 What it does: Propose an Incomplete Utterance Rewriting (IUR) method based on a single-layer MLP, which predicts token-level edit types by leveraging a joint feature matrix of context and incomplete utterances to rapidly generate complete utterances.
HuCurl: Human-induced Curriculum Discovery
Mohamed Elgaar (University of Massachusetts Lowell), Hadi Amiri (University of Massachusetts Lowell)
Hyperparameter SearchData-Centric LearningTransformerText
🎯 What it does: This paper proposes a curriculum discovery framework based on human-annotated entropy or model loss, which can automatically search and discover diverse training curricula on a given model and dataset.
Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition
Yuwei Bao (University of Michigan), Joyce Chai (University of Michigan)
Representation LearningTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Propose a comparison learning framework inspired by human learning to achieve continuous word acquisition and representation learning.
Human-in-the-loop Evaluation for Early Misinformation Detection: A Case Study of COVID-19 Treatments
Ethan Mendes (Georgia Institute of Technology), Alan Ritter (Georgia Institute of Technology)
ClassificationAnomaly DetectionTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Constructed a human-machine collaborative early rumor detection evaluation framework and implemented case studies on COVID-19 treatment misinformation.
Hybrid Knowledge Transfer for Improved Cross-Lingual Event Detection via Hierarchical Sample Selection
Luis Guzman Nateras (University of Oregon), Thien Nguyen (University of Oregon)
Domain AdaptationKnowledge DistillationRepresentation LearningTransformerText
🎯 What it does: A hybrid knowledge transfer framework combining direct transfer and data transfer, utilizing teacher-student distillation for cross-lingual event detection.
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks
Yun Tang (Carnegie Mellon University), Juan Pino (Meta AI)
RecognitionRecurrent Neural NetworkTransformerAudio
🎯 What it does: Proposed a hybrid model (TAED) that combines Transducer with Attention Encoder-Decoder (AED), sharing a speech encoder and replacing the Transducer's predictor with an AED decoder;
Hybrid Uncertainty Quantification for Selective Text Classification in Ambiguous Tasks
Artem Vazhentsev (AIRI), Artem Shelmanov (London Institute for Mathematical Sciences)
ClassificationTransformerText
🎯 What it does: Propose a hybrid uncertainty quantification method (HUQ), combining the likelihood distribution of prior knowledge (epistemic) with entropy related to softmax (aleatoric), to enhance selective classification performance in text classification tasks with high ambiguity (e.g., toxicity detection).
HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbation
Hongyi Yuan (Tsinghua University), Songfang Huang (Alibaba Group)
Representation LearningTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose HyPe, a fine-tuning technique that injects random noise between Transformer layers.
HyperMixer: An MLP-based Low Cost Alternative to Transformers
Florian Mai (Idiap Research Institute), James Henderson (Idiap Research Institute)
Computational EfficiencyRepresentation LearningTransformerTextBenchmark
🎯 What it does: This paper proposes a full MLP architecture called HyperMixer to replace the self-attention module in Transformers.
I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons
Pei Zhou (Allen Institute for Artificial Intelligence), Prithviraj Ammanabrolu (Allen Institute for Artificial Intelligence)
GenerationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Propose the G4C task, learning how the teacher (Dungeon Master) guides the student (player) to complete objectives through natural language interaction in the Dungeons and Dragons (D&D) environment.
I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation
Chandra Bhagavatula (Allen Institute for AI), Yejin Choi (University of Washington)
GenerationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper proposes the I2D2 framework, which uses GPT-2 XL to automatically generate commonsense generic statements.
Ideology Prediction from Scarce and Biased Supervision: Learn to Disregard the “What” and Focus on the “How”!
Chen Chen (Chinese University of Hong Kong), Venkatesh Saligrama (Boston University)
ClassificationAuto EncoderText
🎯 What it does: Propose a novel supervised learning framework that can predict the political ideology of text under conditions of label scarcity and bias, while achieving robust prediction for out-of-distribution inputs.
IDRISI-RA: The First Arabic Location Mention Recognition Dataset of Disaster Tweets
Reem Suwaileh (Qatar University), Tamer Elsayed (Qatar University)
RecognitionTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper constructs the first Arabic disaster-related tweet location extraction dataset, IDRISI-RA, and provides both gold standard and silver standard versions.
IM-TQA: A Chinese Table Question Answering Dataset with Implicit and Multi-type Table Structures
Mingyu Zheng (Chinese Academy of Sciences), Weiping Wang (Baidu Inc)
Graph Neural NetworkTabularBenchmark
🎯 What it does: Constructed the IM-TQA dataset to address implicit and multi-type table structures in table question answering, and proposed the RGCN-RCI two-stage framework for table structure recognition and answer prediction.
Improved Instruction Ordering in Recipe-Grounded Conversation
Duong Le, Alan Ritter (Georgia Institute of Technology)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This study focuses on instruction-based dialogues in the cooking domain, analyzes instruction order errors in GPT-J generated text, and proposes two auxiliary subtasks: user intent detection and instruction state tracking, aiming to improve the accuracy of instruction order in response generation.
Improving Automatic Quotation Attribution in Literary Novels
Krishnapriya Vishnubhotla (University of Toronto), Adam Hammond (University of Toronto)
ClassificationRecognitionRecurrent Neural NetworkTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper modularly decomposes the citation attribution task in literary novels based on the Project Dialogism Novel Corpus, separately evaluating four subtasks: character identification, coreference resolution, quote identification, and speaker attribution, and proposes an improved speaker attribution model on this basis.
Improving Continual Relation Extraction by Distinguishing Analogous Semantics
Wenzheng Zhao (Nanjing University), Wei Hu (Nanjing University)
ClassificationKnowledge DistillationRepresentation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose a novel continuous relation extraction model to address the forgetting of semantically similar relations and sample overfitting issues.
Improving Domain Generalization for Prompt-Aware Essay Scoring via Disentangled Representation Learning
Zhiwei Jiang (Nanjing University), Qing Gu (Nanjing University)
ClassificationRepresentation LearningTransformerPrompt EngineeringContrastive LearningText
🎯 What it does: Proposed a prompt-aware neural automatic essay scoring model called PANN, and designed a decoupled representation learning framework (DRL) (including contrastive regularization with quality/content normalization and counterfactual self-training that removes quality-prompt dependencies), to enhance the generalization ability of automatic essay scoring in unseen prompt (prompt-generalized) scenarios.
Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality
Tanay Dixit (Indian Institute of Technology Madras), Muhao Chen (University of Southern California)
GenerationTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: This paper proposes the EFACTSUM method, which generates multiple candidate summaries and performs dual sorting based on factualness and summary quality. During training, contrastive learning is used to enhance the factualness of abstractive summaries while maintaining or even improving summary quality.
Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting
Zahra Fatemi (University of Illinois Chicago), Caimming Xiong
Computational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: In the paper, the authors propose a new method called GEEP to mitigate gender bias while preserving the knowledge of pre-trained language models;
Improving Generalization in Language Model-based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques
Daking Rai (George Mason University), Ziyu Yao (MIT)
TransformerSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Propose two simple techniques to enhance the generalization ability of text-to-SQL semantic parsing based on pre-trained language models: ① Preprocess input tokens to maintain semantic boundaries in subword tokenization; ② Insert special token markers in inputs and outputs to indicate semantic component boundaries aligned with natural language.
Improving Gradient Trade-offs between Tasks in Multi-task Text Classification
Heyan Chai (Harbin Institute of Technology), Qing Liao (Harbin Institute of Technology)
ClassificationOptimizationConvolutional Neural NetworkText
🎯 What it does: Proposed a gradient trading method called GetMTL to address task conflict problems in multi-task text classification.
Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints
Chao Lou (ShanghaiTech University), Kewei Tu (ShanghaiTech University)
Computational EfficiencyText
🎯 What it does: This paper proposes two low-rank variants of Neural QCFG (E model and P model) to accelerate inference and reduce memory consumption.
Improving Low-resource Named Entity Recognition with Graph Propagated Data Augmentation
Jiong Cai (ShanghaiTech University), Kewei Tu (Alibaba Group)
RecognitionData-Centric LearningTransformerTextGraph
🎯 What it does: Propose a graph propagation-based data augmentation framework GPDA, using natural language text to expand the training set for low-resource named entity recognition
Improving Pretraining Techniques for Code-Switched NLP
Richeek Das (Indian Institute of Technology Bombay), Preethi Jyothi (Deepmind)
Representation LearningTransformerText
🎯 What it does: Proposes improved pre-training methods, including SWITCHMLM that utilizes language boundary information, FREQMLM based on frequency, and RESBERT that incorporates residual connections and auxiliary LID loss in the pre-training model, aiming to enhance the pre-training effectiveness for code-switched text.
Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learning
Ran Zhou (DAMO Academy, Alibaba Group), Chunyan Miao (Nanyang Technological University)
RecognitionTransformerContrastive LearningText
🎯 What it does: Propose the ContProto framework, combining contrastive learning and prototype learning to enhance self-training effectiveness in cross-lingual NER
Improving Syntactic Probing Correctness and Robustness with Control Tasks
Weicheng Ma (Computer Science Dartmouth College), Soroush Vosoughi (Computer Science Dartmouth College)
Explainability and InterpretabilityRepresentation LearningTransformerText
🎯 What it does: Propose two control tasks (random word replacement and random label matching) to reduce co-occurrence memory bias in syntactic probing of pre-trained language models, thereby improving the accuracy, robustness, and consistency of syntactic probing.
Improving the Detection of Multilingual Online Attacks with Rich Social Media Data from Singapore
Janosch Haber (Queen Mary University London), Paul Röttger (University of Oxford)
Domain AdaptationAnomaly DetectionTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper constructs the first SOA dataset targeting multilingual online attacks in Singapore, containing code-mixed texts in Indonesian, Malay, Singlish, etc., and provides fine-grained hierarchical labels and rich metadata; it also presents baseline models and domain adaptation experiments.
Improving the robustness of NLI models with minimax training
Michalis Korakakis (University of Cambridge), Andreas Vlachos (University of Cambridge)
ClassificationAdversarial AttackTransformerText
🎯 What it does: Train an adversarial minimax framework that alternately optimizes the learner and auxiliary model to dynamically adjust the weights of 'hard examples', reducing the NLI model's reliance on shortcuts.
Improving the Robustness of Summarization Systems with Dual Augmentation
Xiuying Chen (KAUST), Xiangliang Zhang (University of Notre Dame)
GenerationAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: This study first evaluates the robustness of summary models against word-level perturbations and identifies their vulnerability through adversarial attacks; subsequently, a dual data augmentation strategy is proposed, significantly enhancing the model's stability on noisy and adversarial samples.
Improving Translation Quality Estimation with Bias Mitigation
Hui Huang (Harbin Institute of Technology), Tiejun Zhao (Toshiba (China) Co Ltd)
Knowledge DistillationTransformerContrastive LearningText
🎯 What it does: This paper proposes a contrastive regularization method for bias mitigation in translation quality estimation models, which constructs negative samples using noisy target sentences to enhance the model's focus on bilingual semantic alignment.
In and Out-of-Domain Text Adversarial Robustness via Label Smoothing
Yahan Yang (University of Pennsylvania), Insup Lee (University of Pennsylvania)
ClassificationAdversarial AttackTransformerSupervised Fine-TuningText
🎯 What it does: Evaluate the robustness of label smoothing (Label Smoothing) in pre-trained Transformer models (e.g., BERT, dBERT) against text attacks, covering both in-domain and out-of-domain scenarios;