EMNLP 2023 Papers — Page 10
Conference on Empirical Methods in Natural Language Processing · 1047 papers
Target-oriented Proactive Dialogue Systems with Personalization: Problem Formulation and Dataset Curation
Jian Wang (Hong Kong Polytechnic University), Wenjie Li (Hong Kong Polytechnic University)
Data SynthesisTransformerLarge Language ModelAgentic AIText
🎯 What it does: Studied a personalized goal-oriented active dialogue system, and automatically constructed the TOPDIAL dataset using a role-playing large model.
Target-to-Source Augmentation for Aspect Sentiment Triplet Extraction
Yice Zhang (Harbin Insitute of Technology), Ruifeng Xu (Harbin Insitute of Technology)
Data SynthesisTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposed a Target-to-Source data augmentation method: by learning a Transformer-based generator that directly generates new sentences from given labels (aspect-sentiment triplet) and syntactic templates (dependency trees), and using fluency and alignment discriminators within a reinforcement learning framework to provide feedback and optimize the generated results.
Task-Adaptive Tokenization: Enhancing Long-Form Text Generation Efficacy in Mental Health and Beyond
Siyang Liu (University of Michigan), Rada Mihalcea (University of Michigan)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose 'Task-Adaptive Tokenization (TaT)' for the mental health question-answering (PsyQA) task, achieving variable text chunking during generation by constructing task-specific vocabulary, merging with pre-trained vocabulary, and initializing new token embeddings.
Task-Agnostic Low-Rank Adapters for Unseen English Dialects
Zedian Xiao (Stanford University), Diyi Yang (Stanford University)
Domain AdaptationComputational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose the HyperLoRA method, leveraging expert linguistic knowledge and hypernetworks to generate low-rank LoRA adapters, achieving task-agnostic dialect adaptation.
Task-Level Thinking Steps Help Large Language Models for Challenging Classification Task
Chunhui Du (Shanghai Jiao Tong University), Yaohui Jin (Shanghai Jiao Tong University)
ClassificationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes Task-Level Thinking Steps and a Progressive Revision Framework, which enhance the performance of large language models on various challenging classification tasks by automatically generating and iteratively refining thinking steps through an LLM agent;
TaskDiff: A Similarity Metric for Task-Oriented Conversations
Ankita Bhaumik (Rensselaer Polytechnic Institute), Vatche Isahagian (IBM Research)
RetrievalTransformerContrastive LearningText
🎯 What it does: Propose a task-oriented dialogue similarity measurement method called TaskDiff.
TaskWeb: Selecting Better Source Tasks for Multi-task NLP
Joongwon Kim (University of Washington), Hannaneh Hajishirzi (University of Washington)
Representation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper constructs a large-scale task-to-task transfer benchmark called TASKWEB and proposes a source task selection method called TASKSHOP based on this benchmark, improving the effectiveness of multi-task learning and zero-shot transfer.
TATA: Stance Detection via Topic-Agnostic and Topic-Aware Embeddings
Hans Hanley (Stanford University), Zakir Durumeric (Stanford University)
ClassificationTransformerContrastive LearningText
🎯 What it does: Designed and implemented a stance detection model called TATA that integrates theme-related (TAW) and theme-unrelated (TAG) embeddings, pre-trained using unlabeled news data and synthetic data, ultimately achieving text and topic stance classification in zero-shot and few-shot scenarios.
Taxonomy Expansion for Named Entity Recognition
Karthikeyan K (Duke University), Miguel Ballesteros (AWS AI Labs)
RecognitionKnowledge DistillationTransformerText
🎯 What it does: Propose a method to expand the vocabulary of Named Entity Recognition (NER) using only partially labeled data, called the Partial Label Model (PLM).
TCFLE-8: a Corpus of Learner Written Productions for French as a Foreign Language and its Application to Automated Essay Scoring
Rodrigo Wilkens (UCLouvain), Thomas François (University of Gothenburg)
ClassificationTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed the TCFLE-8 large-scale French learner writing corpus (6,569 essays covering 6 CEFR levels and 8 native languages), providing rich metadata and automated language feature annotations.
Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning
Dong-Ho Lee (University of Southern California), Jay Pujara (University of Southern California)
TransformerLarge Language ModelPrompt EngineeringGraph
🎯 What it does: Achieving temporal knowledge graph (TKG) prediction through in-context learning (ICL) on large language models (LLM) without requiring additional training or explicit knowledge graph structure.
TempTabQA: Temporal Question Answering for Semi-Structured Tables
Vivek Gupta (University of Pennsylvania), Vivek Srikumar (Bloomberg)
Graph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTabularBenchmarkChain-of-Thought
🎯 What it does: This paper proposes a time reasoning question answering task specifically for entity-centered semi-structured tables, constructs the TEMPTABQA dataset covering 90+ domains with 11,454 question-answer pairs, and systematically evaluates the performance of various NLP models on this task.
Text Embeddings Reveal (Almost) As Much As Text
John Morris, Alexander Rush
GenerationRepresentation LearningTransformerLarge Language ModelTextBiomedical DataBenchmark
🎯 What it does: Investigated the reversibility of text embeddings and proposed the Vec2Text method, which iteratively corrects generated text to approach the target embedding, achieving text recovery.
Text encoders bottleneck compositionality in contrastive vision-language models
Amita Kamath (University of California, Los Angeles), Kai-Wei Chang (University of California, Los Angeles)
Representation LearningTransformerLarge Language ModelVision Language ModelAuto EncoderContrastive LearningTextMultimodalityBenchmark
🎯 What it does: By constructing a progressively increasing set of compositional statements, CompPrompts, and corresponding ControlledImCaps image-text pairs, we investigate the expression bottleneck in the single-vector text encoder of vision-language models (e.g., CLIP), and propose to assess the information loss using only a text recovery probe.
Text Fact Transfer
Nishant Balepur (University of Illinois at Urbana-Champaign), Kevin Chang (University of Illinois at Urbana-Champaign)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Propose a text fact transfer task and design the ModQGA framework to modify factual content while preserving the text style
Text Rendering Strategies for Pixel Language Models
Jonas Lotz (University of Copenhagen), Desmond Elliott (Johns Hopkins University)
Computational EfficiencyRepresentation LearningTransformerVision Language ModelAuto EncoderImageText
🎯 What it does: Investigated the impact of different text rendering strategies on model performance in pixel language models (PIXEL) and proposed an improved method based on character bigrams (BIGRAMS) rendering.
Text Representation Distillation via Information Bottleneck Principle
Yanzhao Zhang (Alibaba Group), Pengjun Xie (Alibaba Group)
Knowledge DistillationRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: This paper proposes an information bottleneck principle-based text representation distillation method called IBKD, which transfers the representation knowledge of large pre-trained models to small models, and achieves unsupervised learning followed by supervised fine-tuning through two-stage training.
Text-Transport: Toward Learning Causal Effects of Natural Language
Victoria Lin (Carnegie Mellon University), Eli Ben-Michael (Carnegie Mellon University)
Domain AdaptationExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Studies how to estimate the causal effects of text attributes through distribution migration when the text distribution does not satisfy causal inference assumptions, and proposes the TEXT-TRANSPORT method.
The ACL OCL Corpus: Advancing Open Science in Computational Linguistics
Shaurya Rohatgi (Pennsylvania State University), Min-Yen Kan (National University of Singapore)
ClassificationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextGraph
🎯 What it does: Constructed the ACL OCL corpus, collecting and organizing structured full texts, metadata, charts, and links to the Semantic Scholar knowledge graph from 73,285 papers in the ACL Anthology spanning 1952 to 2022.
The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models
Jingyuan Qi (Amazon Inc.), Lifu Huang (Virginia Tech)
TransformerLarge Language ModelPrompt EngineeringVision Language ModelTextMultimodality
🎯 What it does: Designed a 'Socratic questioning' recursive prompting algorithm, enabling LLMs to decompose and solve complex reasoning tasks through self-asking and self-answering.
The Benefits of Label-Description Training for Zero-Shot Text Classification
Lingyu Gao (Toyota Technological Institute at Chicago), Kevin Gimpel (Toyota Technological Institute at Chicago)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: By constructing a small-scale dataset containing only label descriptions (unrelated to input text), fine-tune pre-trained language models to improve the accuracy of zero-shot text classification.
The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models
Xinyi Chen (University of Amsterdam), Sandro Pezzelle (University of Amsterdam)
Representation LearningTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelTextMultimodalityBenchmark
🎯 What it does: This paper constructs a new multimodal benchmark dataset called BLA to evaluate the ability of pre-trained vision-language models in handling basic language structures (active-passive voice, coordination, relative clauses). Systematic experiments were conducted on models such as CLIP, ViLBERT, LXMERT, BLIP2, and OpenFlamingo under different settings, including zero-shot, fine-tuning, and context learning.
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning
Seungone Kim (KAIST AI), Minjoon Seo (KAIST AI)
TransformerSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: Constructed the COT COLLECTION dataset and fine-tuned Flan-T5 with Chain-of-Thought (CoT) prompts to obtain the CoT-T5 model, significantly improving the performance of small language models on zero-shot and few-shot reasoning tasks.
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models
Aviv Slobodkin (Bar-Ilan University), Shauli Ravfogel (Bar-Ilan University)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper investigates the internal representations of large language models (LLMs) when handling answerable/unanswerable questions, revealing that they implicitly encode answerability information in hidden layers, and enhances unanswerable detection and hallucination control through prompt engineering, beam search adjustment, and linear subspace erasure.
The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining
Ting-Rui Chiang (University of Southern California), Dani Yogatama (University of Southern California)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelTextSequential
🎯 What it does: Investigate whether the distributional assumptions of masked language model pre-training can explain its sample efficiency and generalization advantages in downstream tasks.
The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models
Lovisa Hagström (Chalmers University of Technology), Richard Johansson (Chalmers University of Technology)
RetrievalTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper evaluates the factual consistency of large language models, improves the ParaRel benchmark to ParaRel*, and systematically analyzes the sources and enhancement methods of consistency through scaling and retrieval augmentation experiments on the LLaMA and Atlas models.
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment
Jared Fernandez (Carnegie Mellon University), Emma Strubell (Carnegie Mellon University)
Computational EfficiencyConvolutional Neural NetworkTransformerTextBenchmark
🎯 What it does: Investigated and quantified the 'framework tax' in deep learning frameworks for NLP inference, where framework overhead causes inference latency to not decrease with increases in model computation or hardware improvements.
The neural dynamics of word recognition and integration
Jon Gauthier (Massachusetts Institute of Technology), Roger Levy (Massachusetts Institute of Technology)
RecognitionLarge Language ModelTextMultimodalityBiomedical DataAudio
🎯 What it does: Propose a computational model for auditory word recognition and integration based on Bayesian decision theory, and use it to interpret EEG data in natural auditory scenes.
The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values
Hannah Rose Kirk (University of Oxford), Scott A. Hale (University of Oxford)
Reinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextReview/Survey Paper
🎯 What it does: This paper systematically reviews and evaluates nearly 95 studies on leveraging human feedback to guide large language models (LLMs) toward development aligned with subjective human preferences and values;
The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis
Pranav Narayanan Venkit (Pennsylvania State University), Shomir Wilson (Pennsylvania State University)
Safty and PrivacyTextReview/Survey Paper
🎯 What it does: Conduct a comprehensive survey of 189 peer-reviewed papers, analyzing the current state of sentiment analysis (SA) in terms of definitions, applications, models, datasets, and biases, while proposing an ethical checklist and improvement suggestions.
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
Siru Ouyang (University Of Illinois Urbana Champaign), Jiawei Han (University Of Illinois Urbana Champaign)
TransformerTextChain-of-Thought
🎯 What it does: Perform self-annotation and clustering on the large-scale user-GPT interaction data from ShareGPT, comparing the task domains with those of traditional NLP benchmarks.
The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages
Chiyu Zhang (University of British Columbia), Muhammad Abdul-Mageed (University of British Columbia)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed and publicly released a social pragmatic meaning (SM) evaluation benchmark named SPARROW, which includes 169 datasets across 64 languages, and evaluated the performance of ChatGPT and multilingual large language models (LLMs) on this benchmark.
The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations
Vipula Rawte (University of South Carolina), Amitava Das (University of South Carolina)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper addresses the hallucination phenomenon generated by large language models, constructing a fine-grained definition and evaluation framework, proposing the Hallucination Vulnerability Index (HVI) and the publicly available HILT dataset, and introducing two hallucination mitigation strategies.
TheoremQA: A Theorem-driven Question Answering Dataset
Wenhu Chen (University of Waterloo), Tony Xia (University of California, Los Angeles)
Large Language ModelTextBenchmarkFinance RelatedPhysics RelatedChain-of-Thought
🎯 What it does: Constructed the TheoremQA question-answering dataset based on university-level theorems and evaluated the theorem reasoning capabilities of various large language models.
Theory of Mind for Multi-Agent Collaboration via Large Language Models
Huao Li (University of Pittsburgh), Katia Sycara (Carnegie Mellon University)
TransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringTextChain-of-Thought
🎯 What it does: Design and evaluate zero-shot multi-agent collaboration based on large language models (LLM), investigating their collaborative and theory of mind (ToM) performance in text-based games
This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models
Iker García-Ferrero (University Basque Country), German Rigau (University Basque Country)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed and released a large-scale (approximately 380,000 sentences) benchmark dataset containing negation words, and used this dataset to evaluate the understanding and reasoning capabilities of multiple open-source large language models (LLMs) in handling negated sentences.
This Reads Like That: Deep Learning for Interpretable Natural Language Processing
Claudio Fanconi, Julia Vogt
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper proposes an interpretable prototype network that achieves high performance in text classification tasks by utilizing weighted similarity and posterior word-level explanations;
Three Stream Based Multi-level Event Contrastive Learning for Text-Video Event Extraction
Jiaqi Li (Southeast University), Guilin Qi (Southeast University)
RecognitionConvolutional Neural NetworkTransformerOptical FlowVideoTextMultimodality
🎯 What it does: Proposed a three-stream multimodal event extraction framework called TSEE, integrating text, video appearance, and optical flow motion features.
Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibration
Daniel Deutsch (Google), Markus Freitag (Google)
TextBenchmark
🎯 What it does: This paper addresses the meta-evaluation of machine translation metrics by explicitly considering score ties in statistical analysis and migrating the evaluation from Kendall τ to a pairwise accuracy-based approach.
TIMELINE: Exhaustive Annotation of Temporal Relations Supporting the Automatic Ordering of Events in News Articles
Sarah Alsayyahi (University of Manchester), Riza Batista-Navarro (University of Manchester)
Recurrent Neural NetworkTransformerLarge Language ModelTextBenchmark
🎯 What it does: Developed a new time relation annotation scheme and generated the TIMELINE corpus, fully annotating all events and their time relations (including non-verb events and long-distance cross-paragraph relations) in news articles;
TLM: Token-Level Masking for Transformers
Yangjun Wu (Zhejiang University), Gang Chen (Zhejiang University)
ClassificationRepresentation LearningTransformerText
🎯 What it does: This paper proposes a Token-Level Masking (TLM) training strategy, which randomly masks tokens in the Transformer self-attention layer. By forcing the model to rely on neighbor information during training to generate robust representations, it achieves a regularization effect.
To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing
Sireesh Gururaja (Carnegie Mellon University), Emma Strubell (Carnegie Mellon University)
TextReview/Survey Paper
🎯 What it does: Conduct long interviews with the NLP research community and perform quantitative analysis of ACL Anthology, systematically organizing the evolution of paradigm shifts, benchmark culture, and software centralization.
To Split or Not to Split: Composing Compounds in Contextual Vector Spaces
Chris Jenkins (University of Stuttgart), Sabine Schulte im Walde (University of Stuttgart)
Representation LearningTransformerTextTime Series
🎯 What it does: Investigated the impact of subword tokenization on the semantic representations of German noun compounds and improved the BERT model by pre-splitting compounds.
TOD-Flow: Modeling the Structure of Task-Oriented Dialogues
Sungryull Sohn (LG AI Research), Honglak Lee (LG AI Research)
GenerationGraph Neural NetworkText
🎯 What it does: This paper proposes the TOD-Flow graph model, which automatically learns task structures from dialog data with dialog behavior annotations and uses them as conditions for any dialog policy or end-to-end generation model to improve dialog behavior prediction and response quality.
Token Prediction as Implicit Classification to Identify LLM-Generated Text
Yutian Chen (Carnegie Mellon University), Bhiksha Raj (Carnegie Mellon University)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed and implemented a new method based on T5, reformulating the multi-class task of identifying the source of LLM-generated text as a next-token prediction task. The method directly outputs a dedicated label token by fine-tuning T5, achieving text source identification without requiring an additional classifier.
ToolWriter: Question Specific Tool Synthesis for Tabular Data
Carlos Gemmell (University of Glasgow), Jeff Dalton
AI Code AssistantTransformerLarge Language ModelTabular
🎯 What it does: This paper proposes ToolWriter, a framework based on tool detection and program generation, which dynamically generates query-specific row filtering tools in table question answering to simplify table input;
TopWORDS-Poetry: Simultaneous Text Segmentation and Word Discovery for Classical Chinese Poetry via Bayesian Inference
Changzai Pan (Tsinghua University), Ke Deng (Tsinghua University)
SegmentationText
🎯 What it does: Developed an unsupervised TopWORDS‑Poetry method capable of simultaneously performing word discovery and text segmentation for classical Chinese poetry.
ToViLaG: Your Visual-Language Generative Model is Also An Evildoer
Xinpeng Wang (Tongji University), Xing Xie (Microsoft Research Asia)
GenerationSafty and PrivacyTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper systematically studies the potential generation of toxic content (including insults, violence, pornography, etc.) by visual-language generation models (VLGMs) and proposes corresponding evaluation and detoxification methods.
Toward a Critical Toponymy Framework for Named Entity Recognition: A Case Study of Airbnb in New York City
Mikael Brunila (McGill University), Grant McKenzie (McGill University)
RecognitionRepresentation LearningTransformerSupervised Fine-TuningTextTabular
🎯 What it does: Built and trained a named entity recognition model based on Airbnb listings to reveal the relationship between the use of place names in NYC Airbnb descriptions and urban spatial patterns (such as gentrification).
Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performance
Shaomu Tan (University of Amsterdam), Christof Monz (University of Amsterdam)
GenerationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Systematically evaluated and quantified performance differences of zero-shot neural machine translation (NMT) across 40 languages and 1,560 translation directions, analyzing the main factors causing these differences.
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
Yifan Hou (ETH Zürich), Mrinmaya Sachan (EPFL)
Explainability and InterpretabilityTransformerText
🎯 What it does: By designing a Mechanistic Probe to recover reasoning trees from the model's attention patterns in multi-step reasoning tasks, this study investigates whether language models truly perform multi-step reasoning.
Towards a Unified Conversational Recommendation System: Multi-task Learning via Contextualized Knowledge Distillation
Yeongseo Jung (Hong Kong University of Science and Technology), Lei Chen (Hong Kong University of Science and Technology)
GenerationRecommendation SystemKnowledge DistillationGraph Neural NetworkTransformerText
🎯 What it does: Proposes a unified dialogue recommendation system that utilizes context-aware knowledge distillation (ConKD) to achieve multi-task learning, enabling both recommendation and dialogue generation within a single model.
Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model
Leo Z. Liu (University of Washington), Xian Li (Meta AI)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: This paper studies a unified perspective of sparse feed-forward networks (S-FFN) in the pre-training of large language models and proposes a new gating method called Avg-K to enhance model performance.
Towards Building More Robust NER datasets: An Empirical Study on NER Dataset Bias from a Dataset Difficulty View
Ruotian Ma (Fudan University), Xuanjing Huang (Fudan University)
RecognitionData-Centric LearningTransformerText
🎯 What it does: The system measures the learning difficulty of entities and contexts in NER datasets through V-information theory, analyzes the impact of dataset bias on model robustness, and improves model performance on robustness evaluation sets such as OOV and Cross-Category by reconstructing the dataset (increasing low CEIM instances, reducing entity V-information, and enhancing context V-information).
Towards Conceptualization of “Fair Explanation”: Disparate Impacts of anti-Asian Hate Speech Explanations on Content Moderators
Tin Nguyen (University of Maryland), Marine Carpuat (University of Maryland)
Explainability and InterpretabilityTransformerText
🎯 What it does: In this paper, the authors evaluate and compare the impact of two NLP explanation methods (significance heatmaps and inverse causal explanations) on the fairness and effectiveness of content moderators (proxies) in judging anti-Asian hate speech through an online human experiment.
Towards Example-Based NMT with Multi-Levenshtein Transformers
Maxime Bouthors (Sorbonne Université), François Yvon (Sorbonne Université)
GenerationExplainability and InterpretabilityKnowledge DistillationData-Centric LearningTransformerTextRetrieval-Augmented Generation
🎯 What it does: Propose Multi-Levenshtein Transformer (TM-N-LevT), which can simultaneously edit and merge multiple retrieved fuzzy matching sentences, followed by iterative refinement to improve the accuracy and interpretability of machine translation.
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation
Yixin Liu (Yale University), Dragomir Radev (Yale University)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Developed a two-stage automatic summarization evaluation metric based on Atomic Content Units (ACU), and proposed an efficient one-stage metric emphasizing interpretability and computational efficiency.
Towards Interpretable Mental Health Analysis with Large Language Models
Kailai Yang (University of Manchester), Sophia Ananiadou (University of Manchester)
ClassificationRecognitionExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Comprehensively evaluate LLMs in mental health analysis, emotional reasoning, and explainability, explore different prompting strategies, and construct a human-evaluated explainability dataset.
Towards LLM-driven Dialogue State Tracking
Yujie Feng (Hong Kong Polytechnic University), Xiao-Ming Wu (Hong Kong Polytechnic University)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper first systematically evaluates the performance of ChatGPT on multi-domain dialog state tracking (DST) tasks, and proposes an LLM-driven DST framework LDST based on small open-source models (e.g., LLaMA), achieving performance comparable to or even better than ChatGPT through assembly domain-slot instruction tuning and parameter-efficient fine-tuning.
Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models
Weishi Wang (Salesforce AI Research), Shafiq Joty (Salesforce AI Research)
Meta LearningAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies low-resource automatic program repair (APR) and proposes the Meta-APR framework, which achieves few-shot error repair by leveraging meta-learning and the pre-trained code model CodeT5.
Towards Noise-Tolerant Speech-Referring Video Object Segmentation: Bridging Speech and Text
Xiang Li (Carnegie Mellon University), Bhiksha Raj (Carnegie Mellon University)
SegmentationTransformerVideoMultimodality
🎯 What it does: Investigate how to migrate existing text-driven reference video object segmentation models to noisy speech inputs, achieving noise-robust speech reference video object segmentation.
Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4
Kellin Pelrine (McGill University), Reihaneh Rabbany (European University Institute)
ClassificationTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper studies the rumor detection task using the large-scale language model GPT-4, focusing on the model's generalization ability, confidence assessment, and performance in multilingual environments.
Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models
Jianwei Li (North Carolina State University), Dongkuan Xu (North Carolina State University)
ClassificationAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: This paper proposes a post-training robust pruning strategy called Ada-Pruning, which enhances the robustness of sparse language models against adversarial attacks by minimizing the reconstruction error between sparse and dense models in the embedding space and feature space, while maximizing the preservation of pre-trained knowledge.
Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents
Jannis Vamvas (University of Zurich), Rico Sennrich (University of Zurich)
RecognitionTransformerLarge Language ModelContrastive LearningTextBenchmark
🎯 What it does: This paper defines the semantic difference identification task as a token-level regression problem and constructs an unsupervised evaluation benchmark across multiple text variants.
Training Simultaneous Speech Translation with Robust and Random Wait-k-Tokens Strategy
Linlin Zhang (Zhejiang University), Zhongqiang Huang (Zhejiang University)
TransformerSupervised Fine-TuningTextMultimodalityAudio
🎯 What it does: This paper proposes a two-stage training framework. It first utilizes the alignment between audio and transcribed text for cross-modal token-level alignment, employing a refined Continuous-Integrate-and-Fire (CIF) pre-trained acoustic encoder. Subsequently, Simultaneous Speech Translation (SimulST) training is conducted on this basis, incorporating a robust random waitk-tokens strategy to enable a single model to balance low latency and high-quality translation.
Transcending Scaling Laws with 0.1% Extra Compute
Yi Tay (Google), Mostafa Dehghani (Google)
Computational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Building upon the existing PaLM model, the authors continue training for a small number of steps (approximately 0.1% additional computation) using UL2's multi-task denoising objective to enhance the model's downstream performance and scalability.
Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models
Pierre Colombo (Equall), Pablo Piantanida (ÉTS Montreal)
ClassificationMeta LearningTransformerTextBenchmark
🎯 What it does: This paper studies the problem of text few-shot classification under API closed-model constraints, proposing a parameter-free Fisher-Rao transductive regularization and constructing a benchmark set that covers multiple languages, multiple categories, and has stronger practicality.
Transfer-Free Data-Efficient Multilingual Slot Labeling
Evgeniia Razumovskaia (University of Cambridge), Anna Korhonen (University of Cambridge)
ClassificationTransformerContrastive LearningText
🎯 What it does: Propose a two-stage, English-assistance-free multi-lingual slot labeling method called TWOSL, achieving slot identification in target languages with extremely limited labeled data.
Transformer-based Live Update Generation for Soccer Matches from Microblog Posts
Masashi Oshika (Nagoya University), Koichi Takeda (Nagoya University)
ClassificationGenerationTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Developed a T5-based system capable of generating real-time updates for football matches from real-time Weibo tweets.
Translating away Translationese without Parallel Data
Rricha Jalota (Saarland University), Josef van Genabith (Saarland University)
GenerationTransformerLarge Language ModelText
🎯 What it does: Research on how to reduce translationese in translated texts through monolingual style transfer.
TRAVEL: Tag-Aware Conversational FAQ Retrieval via Reinforcement Learning
Yue Chen (Ant Group), Wenqiang Lei (Ant Group)
RetrievalGraph Neural NetworkReinforcement LearningTextFinance Related
🎯 What it does: This paper proposes a conversation-based FAQ retrieval framework called TRAVEL based on reinforcement learning, which utilizes tag information to reduce the interference of user click noise on retrieval, thereby accurately locating the user's intent FAQ questions within the fewest rounds.
Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models
Gangwoo Kim (Korea University), Jaewoo Kang (Korea University)
RetrievalTransformerPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the TREE OF CLARIFICATIONS (TOC) framework, which utilizes retrieval-enhanced LLMs to generate multiple clarification forms for ambiguous questions through a recursive tree structure, ultimately producing a comprehensive long-answer that covers all clarification outcomes.
Tree Prompting: Efficient Task Adaptation without Fine-Tuning
Chandan Singh (Microsoft Research), Yuntian Deng (Harvard University)
ClassificationLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a method called Tree Prompting that constructs a decision tree to guide language models (LMs) in performing classification tasks without fine-tuning the LM.
TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models
Jing Xiong (Shenzhen Campus of Sun Yat-Sen University), Qun Liu (Huawei Noah's Ark Lab)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Propose the TRIGO dataset, focusing on formal proofs for simplifying trigonometric expressions, combining manual annotations and auto-generated content, verified in the Lean environment, and subsequently evaluating the proof capabilities of generative language models on this benchmark.
trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback
Alexander Havrilla (CarperAI), Louis Castricato (vectorshift.ai)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposed and implemented the open-source framework trlX for fine-tuning large language models with over 70B parameters using reinforcement learning from human feedback (RLHF), supporting online algorithms (PPO, A2C) and offline algorithms (ILQL), and compatible with various parallel training strategies;
TrojanSQL: SQL Injection against Natural Language Interface to Database
Jinchuan Zhang (Institute of Information Engineering Chinese Academy of Sciences), Songlin Hu (Institute of Information Engineering Chinese Academy of Sciences)
Adversarial AttackTransformerPrompt EngineeringTextTabular
🎯 What it does: This paper proposes TrojanSQL, a backdoor SQL injection attack framework targeting natural language database interfaces (NLIDB), which exploits training data poisoning and prompt engineering to mislead text-to-SQL parsers into generating malicious SQL statements.
TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models
Zorik Gekhman (Technion Israel Institute of Technology), Idan Szpektor (Google Research)
Data SynthesisKnowledge DistillationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Generate realistic model summaries using multiple summarization models, and use a large language model (FLAN-PaLM 540B) to annotate factual consistency for these summaries, constructing 1.4M synthetic training samples to train a student model for evaluating the factual consistency of summaries.
Turn-Level Active Learning for Dialogue State Tracking
Zihan Zhang (University of Technology Sydney), Mohammad-Reza Namazi-Rad (University of Wollongong)
Data-Centric LearningTransformerText
🎯 What it does: Proposes a turn-level active learning framework for dialogue state tracking (DST), actively selecting the most valuable dialogue turns for annotation, significantly reducing annotation costs.
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers
Jon Saad-Falcon (Stanford University), Christopher Potts (Stanford University)
RetrievalDomain AdaptationKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose the UDAPDR method, which leverages LLMs to generate synthetic queries, trains a multi-stage re-ranking model, and distills it into a single efficient retriever to achieve unsupervised domain adaptation.
ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervision
Anastasiia Sedova (University of Vienna), Benjamin Roth (University of Vienna)
ClassificationData-Centric LearningTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: Propose an unsupervised weakly supervised label function correction method named ULF, which utilizes k-fold cross-validation to remove noise from label functions and re-estimate the correspondence between label functions and classes, thereby generating cleaner weak labels.
Uncertainty Guided Global Memory Improves Multi-Hop Question Answering
Alsu Sagirova (Moscow Institute of Physics and Technology), Mikhail Burtsev (London Institute for Mathematical Sciences)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose an uncertainty-guided global explicit memory (GEMFormer) to improve multi-hop question answering (MHQA), by aggregating low-entropy (high-confidence) document tokens into a memory sequence, which is then combined with local context and input into a Transformer for reasoning.
Understanding Compositional Data Augmentation in Typologically Diverse Morphological Inflection
Farhan Samir (University of British Columbia), Miikka Silfverberg (University of British Columbia)
Data SynthesisExplainability and InterpretabilityRepresentation LearningData-Centric LearningTransformerText
🎯 What it does: This paper clarifies how the STEMCORRUPT data augmentation method eliminates pseudo-correlations between stems and inflectional affixes, thereby enhancing combinatorial generalization in morphological inflection tasks, through information-theoretic analysis and experiments. It further explores how combining diversity and predictive uncertainty in subset sampling strategies can improve sample efficiency across multiple languages.
Understanding Computational Models of Semantic Change: New Insights from the Speech Community
Filip Miletić (University of Stuttgart), Ludovic Tanguy (University of Toulouse)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: This study combines a computational semantic change model with empirical data from the Quebec English-speaking community to explore the association between model outputs and community members' perceptions.
Understanding the Effect of Model Compression on Social Bias in Large Language Models
Gustavo Gonçalves (Carnegie Mellon University), Emma Strubell (Carnegie Mellon University)
Knowledge DistillationTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper evaluates the performance of compressed models on social bias after dynamically applying Post-Training Quantization (PTQ) and knowledge distillation to three types of LLMs: BERT, RoBERTa, and Pythia.
Understanding the Inner-workings of Language Models Through Representation Dissimilarity
Davis Brown (Pacific Northwest National Laboratory), Henry Kvinge (Pacific Northwest National Laboratory)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: This paper analyzes the internal representations of language models using representation dissimilarity methods (model stitching and CKA), exploring the asymmetry of different activation functions, identifying differences in hidden layers that lead to generalization failure, and studying changes in hidden features across the Pythia scale family.
Understanding the Role of Input Token Characters in Language Models: How Does Information Loss Affect Performance?
Ahmed Alajrami (University of Sheffield), Nikolaos Aletras (University of Sheffield)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper explores the impact of information loss on model performance by restricting the input of pre-trained language models to subsets of single-character, two-character, or three-character components (including initial characters, final characters, middle characters, vowels, or consonants) of each word, and conducts fine-tuning and evaluation on GLUE/SuperGLUE tasks and six probing tasks.
UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning
Ahmed Masry (York University), Shafiq Joty (Nanyang Technological University)
GenerationKnowledge DistillationTransformerVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Propose UniChart, an end-to-end chart pre-training model that combines an image encoder (Donut + Swin Transformer) and a text decoder (BART), and learns the visual, textual, and numerical reasoning capabilities of charts through multi-task pre-training.
Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning
Sarkar Snigdha Sarathi Das (Pennsylvania State University), Rui Zhang (Pennsylvania State University)
ClassificationTransformerSupervised Fine-TuningText
🎯 What it does: Propose a sample-aware dynamic sparse fine-tuning strategy called FISH-DIP for unified low-resource sequence labeling tasks.
Unifying Cross-Lingual Transfer across Scenarios of Resource Scarcity
Alan Ansell (University of Cambridge), Edoardo Ponti (University of Edinburgh)
ClassificationDomain AdaptationRepresentation LearningTransformerSupervised Fine-TuningText
🎯 What it does: Proposes a unified cross-lingual transfer framework that integrates parameter-efficient language adaptation, few-shot learning, and machine translation training, conducting systematic experiments across different resource-scarce scenarios;
Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation
Mingfeng Xue (Sichuan University), Jiancheng Lv (Sichuan University)
GenerationRepresentation LearningTransformerLarge Language ModelAuto EncoderText
🎯 What it does: Propose an unsupervised synonymous sentence generation method, achieving diverse and entity-accurate rewrites through self-supervised pseudo data construction and unified discrete-continuous representations.
UniMath: A Foundational and Multimodal Mathematical Reasoner
Zhenwen Liang (University of Notre Dame), Xiangliang Zhang (University of Notre Dame)
TransformerLarge Language ModelSupervised Fine-TuningVision Language ModelAuto EncoderImageTextMultimodalityTabularChain-of-Thought
🎯 What it does: A unified mathematical reasoning model named UniMath was constructed, capable of handling multimodal mathematical problems involving text, tables, and geometric images, supporting complete reasoning from problem description to answer;
Universal Self-Adaptive Prompting
Xingchen Wan (Google), Tomas Pfister (Google)
ClassificationGenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Propose an automatic prompt design method called Universal Self-Adaptive Prompting (USP), which generates pseudo-demos (pseudo-demos) using unlabeled data in zero-shot or few-shot scenarios, thereby improving the zero-shot reasoning and generation performance of large language models (LLMs).
Unlearn What You Want to Forget: Efficient Unlearning for LLMs
Jiaao Chen (Georgia Institute of Technology), Diyi Yang (Stanford University)
Safty and PrivacyComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed an efficient LLM unlearning framework called EUL, which can delete specified data through a lightweight unlearning layer without requiring complete retraining;
Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text
Qi Cao (University of Tokyo), Yusuke Iwasawa (University of Tokyo)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Investigate the robustness of LLMs under character-level scrambled text, propose the Scrambled Bench test suite, which includes two tasks: sentence recovery (ScrRec) and question answering (ScrQA); evaluate the performance of LLMs on three major datasets.
Unraveling Feature Extraction Mechanisms in Neural Networks
Xiaobing Sun (Singapore University of Technology and Design), Wei Lu (Singapore University of Technology and Design)
ClassificationExplainability and InterpretabilityRepresentation LearningConvolutional Neural NetworkRecurrent Neural NetworkTransformerTextSequentialBenchmark
🎯 What it does: Studied the theoretical analysis of the feature extraction mechanism of basic NLP models during gradient descent under infinitely wide networks using the Neural Tangent Kernel (NTK).
Unsupervised Grammatical Error Correction Rivaling Supervised Methods
Hannan Cao (National University of Singapore), Hwee Tou Ng (National University of Singapore)
GenerationData SynthesisTransformerLarge Language ModelText
🎯 What it does: This paper proposes a fully unsupervised grammar error correction system that utilizes the BIFI framework to generate synthetic pairs through a masked language model and constructs a discriminator to iteratively enhance the corrector.
Unsupervised Sounding Pixel Learning
Yining Zhang (UESTC), Yang Yang (UESTC)
SegmentationConvolutional Neural NetworkContrastive LearningMultimodality
🎯 What it does: Propose the USPL method to achieve unsupervised pixel-level sound source localization. First, coarse localization is obtained by aligning audio-visual features through Mask Augmentation-based multi-instance contrastive learning. Then, the SMR module refines the boundaries by leveraging visual semantic affinity. Finally, the SPS lightweight segmentation network generates the final result.
Unveiling the Essence of Poetry: Introducing a Comprehensive Dataset and Benchmark for Poem Summarization
Ridwan Mahbub (Islamic University of Technology), Sabbir Ahmed (Islamic University of Technology)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose the poetry summarization task, construct the PoemSum dataset, and benchmark existing summarization models.
Unveiling the Implicit Toxicity in Large Language Models
Jiaxin Wen (Tsinghua University), Minlie Huang (Tsinghua University)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: This paper investigates whether large language models (LLMs) can generate implicit harmful language that is difficult for existing toxicity detectors to capture, and proposes a reinforcement learning-based attack method to make LLMs more likely to generate such implicit harmful outputs.
UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation
Daixuan Cheng (Microsoft Corporation), Qi Zhang (Microsoft Corporation)
RetrievalComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Propose UPRISE, a lightweight retrieval-based prompter that enhances the zero-shot performance of LLMs by training a retriever on a small LLM to retrieve prompts from a pre-built pool that are applicable to any zero-shot task.