ACL 2023 Papers — Page 10
Annual Meeting of the Association for Computational Linguistics · 1074 papers
Target-Side Augmentation for Document-Level Machine Translation
Guangsheng Bao (Zhejiang University), Yue Zhang (Nanyang Technological University)
GenerationData SynthesisTransformerTextBenchmark
🎯 What it does: In document-level machine translation, a target-side data augmentation (target-side augmentation) method is introduced: first, a DA model estimates the posterior distribution and generates multiple possible translations under the condition of a given source document and an observed translation; subsequently, these translations are combined with the source document as training data, and the Transformer or G-Transformer is used for model training, thus obtaining a smoother and more robust translation model.
Targeted Data Generation: Finding and Fixing Model Weaknesses
Zexue He (UC San Diego), Fereshte Khani (Microsoft)
ClassificationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposes the Targeted Data Generation (TDG) framework, which automatically identifies model weaknesses on specific subgroups and leverages large language models (LLMs) combined with human annotations to generate targeted data, thereby improving subgroup performance while maintaining overall accuracy.
TART: Improved Few-shot Text Classification Using Task-Adaptive Reference Transformation
Shuo Lei (Virginia Tech), Chang-Tien Lu (Virginia Tech)
ClassificationRecurrent Neural NetworkText
🎯 What it does: Proposed the TART network, which uses task-adaptive reference transformation to enhance few-shot text classification.
Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering
Hao Cheng (Microsoft Research), Jianfeng Gao (Microsoft Research)
RetrievalComputational EfficiencyTransformerMixture of ExpertsContrastive LearningText
🎯 What it does: This paper proposes a dense retrieval architecture named TASER, achieving parameter sharing by alternately using shared layers and task-specific layers within a single encoder.
TAVT: Towards Transferable Audio-Visual Text Generation
Wang Lin (Zhejiang University), Zhou Zhao (Zhejiang University)
GenerationDomain AdaptationMeta LearningTransformerVision Language ModelContrastive LearningVideoTextMultimodalityBenchmarkAudio
🎯 What it does: Proposed a transferable audio-visual text generation framework called TAVT to address the problem of multimodal domain transfer.
Teaching Small Language Models to Reason
Lucie Charlotte Magister (University of Cambridge), Aliaksei Severyn (Google Research)
Knowledge DistillationTransformerSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: This paper studies transferring the chain-of-thought (CoT) reasoning ability of large language models to smaller models by fine-tuning student models on chain-of-thought data generated by the teacher model to improve reasoning performance.
TeAST: Temporal Knowledge Graph Embedding via Archimedean Spiral Timeline
Jiang Li (Inner Mongolia University), Guanglai Gao (Inner Mongolia University)
Representation LearningGraph Neural NetworkGraphBenchmark
🎯 What it does: Designed a new temporal knowledge graph embedding model called TeAST, which maps relationships to an Archimedean spiral timeline to avoid mixing temporal information with entities, thereby better capturing patterns of how relationships evolve over time; and converts the quadruple completion problem into a third-order tensor completion task.
TECHS: Temporal Logical Graph Networks for Explainable Extrapolation Reasoning
Qika Lin (Xi'an Jiaotong University), Erik Cambria (Nanyang Technological University)
Explainability and InterpretabilityComputational EfficiencyRecurrent Neural NetworkGraph Neural NetworkGraph
🎯 What it does: Proposed an explainable temporal logic graph network (TECHS) framework for predicting future facts based on past facts.
TeCS: A Dataset and Benchmark for Tense Consistency of Machine Translation
Yiming Ai (Shanghai Jiao Tong University), Rui Wang (Shanghai Jiao Tong University)
Convolutional Neural NetworkRecurrent Neural NetworkTransformerTextBenchmark
🎯 What it does: Proposed a tense consistency test set and evaluation benchmark for French-English machine translation
Tell2Design: A Dataset for Language-Guided Floor Plan Generation
Sicong Leng (Singapore University of Technology and Design), Wei Lu (Singapore University of Technology and Design)
GenerationTransformerVision Language ModelDiffusion modelGenerative Adversarial NetworkImageTextMultimodality
🎯 What it does: Proposed a language-guided floor plan generation task and constructed a large-scale Tell2Design dataset
TemplateGEC: Improving Grammatical Error Correction with Detection Template
Yinghao Li (Beijing Institute of Technology), Min Zhang (Harbin Institute of Technology)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper proposes a Chinese and English grammar error correction method called TemplateGEC, which combines Seq2Edit for error detection and Seq2Seq for error correction, and enhances robustness through consistency learning.
Test-time Adaptation for Machine Translation Evaluation by Uncertainty Minimization
Runzhe Zhan (University of Macau), Min Zhang (Harbin Institute of Technology)
Text
🎯 What it does: Propose an adaptive machine translation evaluation method that minimizes uncertainty during testing
Text Adversarial Purification as Defense against Adversarial Attacks
Linyang Li (Fudan University), Xipeng Qiu (Fudan University)
ClassificationAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: This paper proposes a text adversarial purification method based on mask language models, which removes perturbations from word substitution attacks through multiple mask-recovery processes and performs purification before classification.
Text Style Transfer Back-Translation
Daimeng Wei (Huawei Translation Service Center), Hao Yang (Huawei Translation Service Center)
GenerationDomain AdaptationTransformerTextBiomedical Data
🎯 What it does: Propose Text Style Transfer for Back-Translation (TST BT), which improves translation quality for natural inputs by performing style transfer on source-side text from back-translation, making it closer to natural language corpora.
Text Style Transfer with Contrastive Transfer Pattern Mining
Jingxuan Han (University of Science and Technology of China), Zhendong Mao (University of Science and Technology of China)
GenerationTransformerContrastive LearningText
🎯 What it does: This paper proposes a new method for text style transfer tasks—Contrastive Transfer Pattern Mining (CTPM), which automatically clusters to mine hidden transfer patterns and combines contrastive learning to enhance sentence representations, thereby achieving better style control and content preservation.
Text-to-SQL Error Correction with Language Models of Code
Ziru Chen (Ohio State University), Huan Sun (Ohio State University)
Data SynthesisAI Code AssistantTransformerLarge Language ModelTextTabularBenchmark
🎯 What it does: This paper proposes a text-to-SQL error correction model based on clause-level editing and Python dictionary representation.
The Art of Prompting: Event Detection based on Type Specific Prompts
Sijia Wang (Virginia Tech), Lifu Huang (Virginia Tech)
TransformerPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes a unified framework that utilizes multiple prompts for event types (naming, definition, seed triggers, structure, soft prompts, and their comprehensive description APEX) for supervised, few-shot, and zero-shot event detection, and verifies its effectiveness through experiments.
The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers
Ariel Gera (IBM Research), Eyal Shnarch (IBM Research)
GenerationTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose Auto-Contrastive Decoding (ACD), which enhances text generation quality by realigning the probability distribution of the next token through contrasting low-level 'amateur' predictions with high-level 'expert' predictions within the same transformer model.
The Best of Both Worlds: Combining Human and Machine Translations for Multilingual Semantic Parsing with Active Learning
Zhuang Li (Openstream.ai), Gholamreza Haffari (Openstream.ai)
Representation LearningData-Centric LearningRecurrent Neural NetworkTransformerText
🎯 What it does: Proposed a hybrid translation method HAT based on active learning, combining machine translation with a small amount of human translation to train a multilingual semantic parser.
The CRINGE Loss: Learning what language not to model
Leonard Adolphs (Meta AI), Jason Weston (Meta AI)
Safty and PrivacyTransformerContrastive LearningText
🎯 What it does: This paper proposes the CRINGE loss function, which can simultaneously utilize positive and negative samples in language model training to suppress harmful or inconsistent generations.
The Ecological Fallacy in Annotation: Modeling Human Label Variation goes beyond Sociodemographics
Matthias Orlikowski (Bielefeld University), Dirk Hovy (Bocconi University)
ClassificationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Introduces group-specific layers based on socio-demographic attributes into a multi-annotator model to explore the impact of socio-demographic attributes on annotation behavior.
The Elephant in the Room: Analyzing the Presence of Big Tech in Natural Language Processing Research
Mohamed Abdalla (Sorbonne Université), Karen Fort
Text
🎯 What it does: Conduct large-scale industry influence analysis on academic papers in the field of natural language processing (NLP), systematically quantifying and describing the author affiliation, research topics, collaboration patterns, and citation performance of big technology companies (Big Tech). The study also provides a detailed analysis of the interaction between industry and academia by combining manually extracted author resumes.
The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metrics
Ricardo Rei (Unbabel), André Martins
Explainability and InterpretabilityText
🎯 What it does: Investigated the interpretability of neural machine translation evaluation metrics (COMET and UNITE), compared multiple attribution methods, and conducted alignment evaluation with human-annotated MQM error spans.
The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources
Akshatha Arodi (McGill University), Jackie Chi Kit Cheung (Microsoft Research)
Representation LearningRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Designed and constructed the KITMUS coreference evaluation dataset to investigate the model's reasoning ability under knowledge source integration during pre-training and inference.
The Mechanical Bard: An Interpretable Machine Learning Approach to Shakespearean Sonnet Generation
Edwin Agnew (Duke University), Cynthia Rudin (Duke University)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Automatically generate poems that conform to the structure of Shakespearean sonnets (ABAB BCBC CDCD EE, iambic pentameter).
The Role of Global and Local Context in Named Entity Recognition
Arthur Amalvy (Laboratoire Informatique d'Avignon), Richard Dufour (Laboratoire des Sciences du Numérique de Nantes)
RecognitionTransformerSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: The study investigates the role of local and global context in named entity recognition (NER), proposes and evaluates multiple sentence retrieval heuristics and oracle re-ranking methods, and improves a literary text NER dataset;
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks
Nikil Selvam, Kai-Wei Chang (University of California, Los Angeles)
Data-Centric LearningTransformerTextBenchmark
🎯 What it does: Investigate the reliability of social bias benchmarks by introducing shallow data construction variations on WINOGENDER and BIASNLI, and assess their impact on model bias metrics.
Theory-Grounded Computational Text Analysis
Arya D. McCarthy (Johns Hopkins University), Giovanna Maria Dora Dore (Johns Hopkins University)
TextReview/Survey Paper
🎯 What it does: This paper systematically reviews ACL conference papers from the past decade, exploring the shortcomings of computational text analysis in theoretical construction, pointing out that the field overly emphasizes descriptive results and lacks integrative, theory-driven research;
ThinkSum: Probabilistic reasoning over sets using large language models
Batu Ozturkler (Stanford University), Nebojsa Jojic (Microsoft Research)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringMixture of ExpertsTextBenchmarkChain-of-Thought
🎯 What it does: Proposes the ThinkSum two-stage probabilistic reasoning framework: first, use a large language model for rapid retrieval (Think) to generate multiple candidate sets, then complete reasoning in the Sum stage through external probabilistic aggregation (e.g., mixture, product, EM, etc.).
To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering
Dheeru Dua (University of California, Irvine), Pat Verga (Google DeepMind)
Domain AdaptationTransformerLarge Language ModelTextBiomedical DataBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper investigates the generalization and adaptation of open-domain question answering (ODQA) systems under cross-domain (non-conservative) data distribution shifts, constructing seven evaluation datasets spanning five different domains. It proposes an unsupervised generalization testing method to assess the compatibility of source models in target domains. Subsequently, it explores various zero-shot and few-shot data augmentation interventions (including question generation, cloze-style fill-in-the-blank, answer distribution sampling, etc.) and evaluates their impact on end-to-end performance improvements for retrievers and readers.
To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion
Rui Li (Dalian University of Technology), Xing Xie (Microsoft Research Asia)
OptimizationRepresentation LearningGraphBenchmark
🎯 What it does: Proposed the Vertical Learning Paradigm (VLP), which enhances Knowledge Graph Completion (KGC) performance by explicitly copying relevant fact triplets, and designed a negative sampling method based on relative distance (ReD) for more effective optimization.
To Revise or Not to Revise: Learning to Detect Improvable Claims for Argumentative Writing Support
Gabriella Skitalinskaya (Leibniz University Hannover), Henning Wachsmuth (Leibniz University Hannover)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Studied how to identify propositions in argumentative text that require revision and provided suitable revision types.
Token-Level Self-Evolution Training for Sequence-to-Sequence Learning
Keqin Peng, Dacheng Tao (University of Sydney)
GenerationTransformerText
🎯 What it does: This paper proposes a training strategy based on token-level self-evolution (Token-Level Self-Evolution Training, SE) for learning in sequence-to-sequence models;
Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictions
Byung-Doh Oh (Ohio State University), William Schuler (Ohio State University)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Propose a linear decomposition of the hidden states in autoregressive language models, and define ∆LP as an importance metric based on this decomposition to quantify the impact of each context word on the prediction of the next word.
Tokenization and the Noiseless Channel
Vilém Zouhar (ETH Zürich), Ryan Cotterell (ETH Zürich)
Text
🎯 What it does: Investigate the impact of subword tokenization on machine translation performance, propose a metric using Rényi entropy to measure tokenization efficiency, and validate its correlation with BLEU.
TOME: A Two-stage Approach for Model-based Retrieval
Ruiyang Ren (Renmin University of China), Haifeng Wang (Baidu Inc)
RetrievalTransformerLarge Language ModelText
🎯 What it does: Proposed a two-stage generation-based model retrieval framework called TOME, which first generates paragraphs relevant to the query and then generates corresponding URLs as retrieval results based on these paragraphs;
Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection
Erik Arakelyan (University of Copenhagen), Isabelle Augenstein (University of Copenhagen)
ClassificationDomain AdaptationRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: Proposed the TESTED framework, achieving cross-domain stance detection through theme-oriented diverse sampling and contrastive learning;
Toward Expanding the Scope of Radiology Report Summarization to Multiple Anatomies and Modalities
Zhihong Chen (Chinese University of Hong Kong Shenzhen), Jean-Benoit Delbrouck (Stanford University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningMultimodalityBiomedical DataMagnetic Resonance ImagingComputed TomographyElectronic Health Records
🎯 What it does: This study constructs an open-access radiology report summary dataset, MIMIC-RRS, spanning multiple imaging modalities and anatomical regions, and conducts multi-modal summarization benchmark experiments.
Toward Human-Like Evaluation for Natural Language Generation with Error Analysis
Qingyu Lu (Southeast University), Dacheng Tao (Southeast University)
GenerationTransformerLarge Language ModelText
🎯 What it does: Propose BARTScore++ to achieve human-like evaluation through error analysis.
Toward Interactive Dictation
Belinda Z. Li (MIT), Sam Thomson (Microsoft)
TransformerLarge Language ModelTextBenchmarkAudio
🎯 What it does: Developed an interactive speech-to-text system that supports inserting natural language editing commands at any time during voice input.
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review
Fred Philippy (Zortify S.A.), Shohreh Haddadan (Zortify S.A.)
TransformerLarge Language ModelTextReview/Survey PaperBenchmark
🎯 What it does: Summarize and systematically categorize the influencing factors of cross-lingual transfer in multilingual models, constructing a five-factor framework and reconciling contradictions among existing studies.
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning
Zhen-Ru Zhang (Alibaba Group), Songfang Huang (Alibaba Group)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper proposes Adaptive Prefix Tuning (APT), which dynamically adjusts prefix vectors by using fine-grained token-level gating and coarse-grained hierarchical scaling gating within Transformer layers to enhance the effectiveness of parameter-efficient fine-tuning.
Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models
Qingyu Tan (Alibaba Group), Lidong Bing (Alibaba Group)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: This paper constructs TEMPREASON, a time-sensitive question-answering dataset covering three levels: time-time, time-event, and event-event, systematically evaluates the temporal reasoning ability of large language models (LLMs), and proposes improvement methods targeting their weaknesses;
Towards Better Entity Linking with Multi-View Enhanced Distillation
Yi Liu (Institute of Information Engineering, Chinese Academy of Sciences), Qi Zhang (Microsoft)
RetrievalKnowledge DistillationTransformerTextBenchmark
🎯 What it does: This paper proposes the Multi-View Enhanced Distillation (MVD) framework, which transfers multi-view entity representations and the precise learning capabilities of cross-encoders to dual encoders through knowledge distillation, thereby improving the fine-grained matching performance in entity retrieval.
Towards Boosting the Open-Domain Chatbot with Human Feedback
Hua Lu (Baidu), Haifeng Wang (Baidu)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Construct the Diamante Chinese casual chat dataset by collecting human selections/rewrites of model-generated candidate responses, and improve open-domain chatbots through generation-evaluation joint training based on this dataset.
Towards Domain-Agnostic and Domain-Adaptive Dementia Detection from Spoken Language
Shahla Farzana (University of Illinois Chicago), Natalie Parde (University of Illinois Chicago)
Domain AdaptationBiomedical DataAlzheimer's DiseaseAudio
🎯 What it does: Studying how to utilize domain adaptation techniques to improve the accuracy of dementia detection in spoken languages across different domains.
Towards Faithful Dialogues via Focus Learning
Yifan Deng (Chinese Academy of Sciences), Yue Hu (Chinese Academy of Sciences)
GenerationTransformerLarge Language ModelText
🎯 What it does: To address the hallucination problem in knowledge-driven dialogue systems, Focus Learning (FocusL) is proposed—by locating the semantic relevance between each response word and knowledge, mapping this relevance to dynamic weights, and directly scaling the cross-entropy loss, thereby guiding the model to focus on knowledge-related words during training.
Towards Fewer Hallucinations in Knowledge-Grounded Dialogue Generation via Augmentative and Contrastive Knowledge-Dialogue
Bin Sun (Beijing Institute of Technology), Kan Li (Huawei Noah's Ark Lab)
GenerationOptimizationTransformerSupervised Fine-TuningPrompt EngineeringContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Design and implement an Augmentative and Contrastive Knowledge Dialogue Expansion Framework (ACK-DEF), which generates knowledge-dialogue samples with varying levels of noise to smooth the optimization objective and reduce hallucination in knowledge-driven dialogues.
Towards Higher Pareto Frontier in Multilingual Machine Translation
Yichong Huang (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
OptimizationKnowledge DistillationHyperparameter SearchTransformerText
🎯 What it does: Propose the Pareto Mutual Distillation (Pareto-MD) framework, which uses two sets of multilingual translation models trained with different sampling distributions. During training, the models distill knowledge from each other to improve the Pareto frontier of multilingual machine translation.
Towards Identifying Fine-Grained Depression Symptoms from Memes
Shweta Yadav (University of Illinois Chicago), Tanmay Sharma (Indian Institute of Technology Gandhinagar)
ClassificationTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: Constructed the RESTORE high-quality dataset, collected and annotated 9,837 depression-related memes, and proposed an orthogonal constraint-based multimodal model to identify fine-grained depressive symptoms in memes.
Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages
Sumanth Doddapaneni (Indian Institute of Technology Madras), Pratyush Kumar (Indian Institute of Technology Madras)
ClassificationRetrievalTransformerLarge Language ModelTextBenchmark
🎯 What it does: This study constructs the largest Indic corpus, IndicCorp v2 (20.9B words, 24 languages), releases a human-supervised NLU benchmark, IndicXTREME (9 tasks, 105 evaluation sets, 52 of which are newly created), and trains and evaluates IndicBERT v2 (278M parameters) on this benchmark.
Towards Open-World Product Attribute Mining: A Lightly-Supervised Approach
Liyan Xu (WeChat AI), Jinho D. Choi (Emory University)
Data-Centric LearningTransformerContrastive LearningText
🎯 What it does: This paper proposes a lightweight supervised open-world product attribute mining framework called Amacer, which can expand existing attribute types and automatically discover new attribute types with only a small number of seed attribute values.
Towards Robust Low-Resource Fine-Tuning with Multi-View Compressed Representations
Linlin Liu (DAMO Academy, Alibaba Group), Lidong Bing (DAMO Academy, Alibaba Group)
Representation LearningTransformerSupervised Fine-TuningAuto EncoderText
🎯 What it does: When fine-tuning pre-trained language models (PLM), randomly initialized autoencoders (AE) are inserted between different layers. During forward propagation, these AEs are randomly selected and used to compress hidden representations into multi-view compressed representations (MVCR). During inference, these AEs are removed, keeping the original model's parameters and inference cost unchanged.
Towards Stable Natural Language Understanding via Information Entropy Guided Debiasing
Li Du (Harbin Institute of Technology), Jingshuo Liu (Harbin Institute of Technology)
ClassificationRecognitionRepresentation LearningTransformerText
🎯 What it does: Propose an information entropy guided automatic debiasing framework named IEGDB, which enhances the OOD stability of NLU models by utilizing randomly induced bias feature forests and entropy separation.
Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation
Soyoung Yoon (University of California, Santa Barbara), Alice Oh (KAIST)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Collected three Korean grammar error correction (GEC) parallel corpora (Kor-Lang8, Kor-Native, Kor-Learner), and developed an automatic error type annotation tool called KAGAS; subsequently, fine-tuned KoBART to build a baseline model.
Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation
Songming Zhang (Beijing Jiaotong University), Jinan Xu (Beijing Jiaotong University)
Knowledge DistillationTransformerText
🎯 What it does: This paper investigates the knowledge source in knowledge distillation for neural machine translation (NMT), revealing that it primarily comes from the teacher model's top-1 prediction information, and proposes the TIE-KD method, which enhances the student model's performance through hierarchical ranking loss and iterative distillation.
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
Boshi Wang (Ohio State University), Huan Sun (Ohio State University)
Large Language ModelTextChain-of-Thought
🎯 What it does: Investigate the effectiveness of Chain-of-Thought (CoT) prompting by conducting large-scale ablation experiments to analyze how different components of CoT examples (effectiveness, relevance, coherence) influence model reasoning.
Towards Understanding Omission in Dialogue Summarization
Yicheng Zou (Fudan University), Tao Gui (Fudan University)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Investigated the omission problem in dialogue summarization, constructed a high-quality omission label dataset called OLDS, and proposed the omission detection task in dialogue summarization;
Towards Unifying Multi-Lingual and Cross-Lingual Summarization
Jiaan Wang (Soochow University), Jie Zhou (Tencent Inc)
GenerationTransformerLarge Language ModelText
🎯 What it does: Proposed unifying multi-lingual summarization (MLS) with cross-lingual summarization (CLS) into a many-to-many summarization (M2MS) framework, and experimentally validated its effectiveness.
Towards Zero-Shot Multilingual Transfer for Code-Switched Responses
Ting-Wei Wu (Georgia Institute of Technology), Biing Juang (Georgia Institute of Technology)
GenerationDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningAuto EncoderText
🎯 What it does: Propose the XDFusion framework, combining multilingual Seq2seq models with language adapters and fusion modules to achieve zero/few-shot cross-lingual response generation
Tracing Linguistic Markers of Influence in a Large Online Organisation
Prashant Khare (Queen Mary University of London), Matthew Purver (Queen Mary University of London)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: Analyze the relationship between influence and language use in IETF email data; employ LIWC word class statistics and BERT representations to build predictive models and analyze key linguistic features.
Trading Syntax Trees for Wordpieces: Target-oriented Opinion Words Extraction with Wordpieces and Aspect Enhancement
Samuel Mensah (University of Sheffield), Nikolaos Aletras (University of Sheffield)
RecognitionTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper addresses the task of target sentiment word extraction by removing the syntax tree + GCN from the model and instead using BERT subwords (wordpiece) and sentence-target pairs as input, improving aspect representation and achieving precise extraction of target sentiment words.
Training Models to Generate, Recognize, and Reframe Unhelpful Thoughts
Mounica Maddela (Georgia Tech), Y-Lan Boureau (Meta AI)
ClassificationRecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose the PATTERNREFRAME dataset for generating, identifying, and reconstructing unproductive thought patterns;
Training Trajectories of Language Models Across Scales
Mengzhou Xia (Princeton University), Veselin Stoyanov (Meta AI)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Systematically analyzed the training trajectory of the OPT model from 125M to 175B parameters, evaluating intermediate checkpoints on next-token prediction, sequence generation, and downstream task performance on BIG-Bench.
Training-free Neural Architecture Search for RNNs and Transformers
Aaron Serianni (Princeton University), Jugal Kalita (University of Colorado Colorado Springs)
Neural Architecture SearchRecurrent Neural NetworkTransformerTextBenchmark
🎯 What it does: Studied and evaluated training-free NAS metrics for automatic architecture search on RNN and BERT-based Transformer language modeling tasks.
Transfer and Active Learning for Dissonance Detection: Addressing the Rare-Class Challenge
Vasudha Varadarajan (Stony Brook University), H. Andrew Schwartz (Stony Brook University)
ClassificationAnomaly DetectionTransformerText
🎯 What it does: In the rare category (cognitive dissonance) detection task, combine transfer learning and active learning to improve model performance.
Transformed Protoform Reconstruction
Young Min Kim, David R. Mortensen (Carnegie Mellon University)
GenerationTransformerText
🎯 What it does: Proposed a prototype word reconstruction model based on Transformer for inferring ancestral word forms from sub-language speech or spelling.
Transforming Visual Scene Graphs to Image Captions
Xu Yang (Southeast University), Yu Zhang (Southeast University)
GenerationGraph Neural NetworkTransformerMixture of ExpertsMultimodalityGraph
🎯 What it does: Propose an isomorphic Transformer framework to convert visual scene graphs (Scene Graph) into more descriptive image captions;
Translation-Enhanced Multilingual Text-to-Image Generation
Yaoyiran Li (University of Cambridge), Anna Korhonen (University of Cambridge)
GenerationVision Language ModelGenerative Adversarial NetworkContrastive LearningMultimodality
🎯 What it does: This paper studies multilingual text-to-image generation (mTTI), aiming to improve image generation quality for non-English languages through machine translation techniques.
TREA: Tree-Structure Reasoning Schema for Conversational Recommendation
Wendi Li (Huazhong University of Science and Technology), Dangyang Chen (Ping An Property & Casualty Insurance company of China)
Recommendation SystemGraph Neural NetworkTransformerTextGraph
🎯 What it does: Propose a tree-structured reasoning framework called TREA for conversational recommendation systems, which constructs a multi-level scalable reasoning tree to track entity causal relationships and generate recommendations.
Tree-Based Representation and Generation of Natural and Mathematical Language
Alexander Scarlatos (University of Massachusetts Amherst), Andrew Lan (University of Massachusetts Amherst)
GenerationRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose an improved language model MathGPT that can jointly represent and generate natural language and mathematical expressions.
Trigger Warning Assignment as a Multi-Label Document Classification Problem
Matti Wiegmann (Bauhaus-Universität Weimar), Martin Potthast (Leipzig University)
ClassificationTransformerTextBenchmark
🎯 What it does: Treat trigger warning assignment as a multi-label text classification task, constructing the Webis Trigger Warning Corpus 2022, which contains approximately 1.09 M AO3 fanfiction works and provides corresponding 36 classes of trigger warning labels.
Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis
Agam Shah (Georgia Institute of Technology), Sudheer Chava (Georgia Institute of Technology)
ClassificationRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextTabularFinance Related
🎯 What it does: Constructed the largest FOMC text dataset and proposed a three-class hawkish/dovish classification task, utilizing RoBERTa-large to generate policy stance indicators and validate their predictive power for CPI, PPI, treasury yields, and stock markets.
TwistList: Resources and Baselines for Tongue Twister Generation
Tyler Loakman (University of Sheffield), Chenghua Lin (University of Sheffield)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposed the stuttering sentence generation task, released the TwistList dataset (2.1K+ stuttering sentences + keywords + phonetic symbols), and provided a series of baseline models (TwisterMisters).
Two Birds One Stone: Dynamic Ensemble for OOD Intent Classification
Yunhua Zhou (Fudan University), Xipeng Qiu (Fudan University)
ClassificationComputational EfficiencyTransformerText
🎯 What it does: Investigated the 'overthinking' phenomenon in OOD intent classification and proposed a dynamic early-exit strategy to simultaneously improve speed and accuracy.
Two-Stage Fine-Tuning for Improved Bias and Variance for Large Pretrained Language Models
Lijing Wang (New Jersey Institute of Technology), Guergana Savova (Boston Children's Hospital and Harvard Medical School)
OptimizationHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Propose a two-phase fine-tuning method: the first phase evaluates the generalization ability of pre-trained models through variance decomposition and iteratively reduces bias and variance; the second phase further reduces optimization-induced variance using bagging and dynamic snapshot ensembling.
Typo-Robust Representation Learning for Dense Retrieval
Panuthep Tasawong (VISTEC), Sarana Nutanong (VISTEC)
RetrievalRepresentation LearningTransformerLarge Language ModelContrastive LearningTextBenchmark
🎯 What it does: Propose a training framework based on Dual Self-Teaching (DST), enhancing the robustness of dense retrieval models to spelling error queries through query typo generation, bidirectional consistency loss, and contrastive loss.
U-CREAT: Unsupervised Case Retrieval using Events extrAcTion
Abhinav Joshi (Indian Institute of Technology Kanpur), Ashutosh Modi (Indian Institute of Technology Kanpur)
RetrievalTransformerText
🎯 What it does: Proposed a new Indian Legal Precedent Retrieval (IL-PCR) dataset and designed an unsupervised event extraction retrieval framework, U-CREAT, to efficiently retrieve precedent documents relevant to query cases.
UMRSpell: Unifying the Detection and Correction Parts of Pre-trained Models towards Chinese Missing, Redundant, and Spelling Correction
Zheyu He (PingAn OneConnect), Liang Xu (PingAn OneConnect)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes a unified Chinese misspelling correction model called UMRSpell that integrates detection and correction.
Unbalanced Optimal Transport for Unbalanced Word Alignment
Yuki Arase (Osaka University), Sho Yokoi (Tohoku University)
OptimizationText
🎯 What it does: This paper proposes using the optimal transport (BOT, POT, UOT) framework to achieve monolingual unbalanced word alignment, utilizing general cost and allocation for alignment in unsupervised scenarios, and implementing end-to-end training through linear metric learning in supervised scenarios.
Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction
Qi Sun (Nanjing University of Science and Technology), Soujanya Poria (Singapore University of Technology and Design)
Data-Centric LearningGraph Neural NetworkTransformerTextBenchmark
🎯 What it does: Developed a label denoising framework called UGDRE based on uncertainty guidance to improve noisy data in document-level distant relation extraction.
Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Data
Yufei Li (University of California, Riverside), Cong Liu (University of California, Riverside)
Data-Centric LearningTransformerText
🎯 What it does: Proposed an uncertainty-aware bootstrapping learning framework named UnBED for denoising in remote supervision for joint entity and relation extraction.
Uncovering and Categorizing Social Biases in Text-to-SQL
Yan Liu (Microsoft Research), Jian-Guang Lou (Microsoft Research)
TransformerLarge Language ModelSupervised Fine-TuningTextTabularBenchmark
🎯 What it does: Investigated and revealed social bias issues in Text-to-SQL models, constructing the BiaSpider evaluation benchmark.
Understanding and Bridging the Modality Gap for Speech Translation
Qingkai Fang (Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences), Yang Feng (Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences)
TransformerTextMultimodalityAudio
🎯 What it does: Propose an end-to-end speech translation method called CRESS, which leverages multi-task learning to share speech and text models, and bridges the modality gap between speech and text through cross-modal regularization and token-level adaptive training;
Understanding and Improving the Robustness of Terminology Constraints in Neural Machine Translation
Huaao Zhang (RoyalFlush AI Research Institute), Ming Chen (Zhejiang University)
GenerationTransformerTextBenchmark
🎯 What it does: This study investigates the robustness of terminology-constrained methods in neural machine translation (Placeholder PH and Code-Switch CS), finding inconsistent performance based on the quantity and length of constraints; based on this, a Robust Terminology Translation (RTT) method combining the advantages of PH and CS is proposed, along with the creation of a more challenging English-German terminology-constrained test set.
Understanding Client Reactions in Online Mental Health Counseling
Anqi Li (Zhejiang University), Zhenzhong Lan (Westlake University)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Constructed a theory-driven bidirectional annotation framework tailored for online psychological counseling, encompassing counselor intent, strategies, client reactions, and behaviors; annotated 2,382 Chinese text counseling sessions under this framework; conducted quantitative analysis to examine the impact of client reactions on session effectiveness and how counselors adjust strategies based on feedback; trained a multi-task classification model based on RoBERTa-large, achieving automatic annotation of strategies and reactions;
Understanding Demonstration-based Learning from a Causal Perspective
Ruiyi Zhang (Adobe Research), Tong Yu (Adobe Research)
Meta LearningTransformerPrompt EngineeringText
🎯 What it does: The study demonstrates the performance of demonstration learning in few-shot learning and constructs a structural causal model (SCM) to explain how demonstrations affect the model.
Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors
Liyan Tang (University of Texas at Austin), Greg Durrett (University of Texas at Austin)
GenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper aggregates nine existing fact error annotation datasets and stratifies them by the era of generation models (old models, early Transformers, state-of-the-art fine-tuned models) to construct a new AGGREFACT benchmark for systematically evaluating fact consistency detection metrics.
Understanding In-Context Learning via Supportive Pretraining Data
Xiaochuang Han (University of Washington), Tianlu Wang (Meta AI)
Meta LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The study selects a small sample from the OPT pre-training data that supports ICL using a gradient similarity method, and fine-tunes these samples to enhance ICL performance.
UniCoRN: Unified Cognitive Signal ReconstructioN bridging cognitive signals and human language
Nuwa Xi (Harbin Institute of Technology), Ting Liu (Harbin Institute of Technology)
GenerationConvolutional Neural NetworkTransformerAuto EncoderBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes the first open-vocabulary fMRI-to-text decoding task, fMRI2text, and introduces a unified two-stage encoder-decoder framework, UniCoRN, for converting cognitive signals such as fMRI and EEG into natural language;
UniEvent: Unified Generative Model with Multi-Dimensional Prefix for Zero-Shot Event-Relational Reasoning
Zhengwei Tao (Peking University), Chongyang Tao (Microsoft)
GenerationTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: Propose a unified generative model called UNIEVENT, which can perform multiple event relation reasoning tasks in zero-shot scenarios;
UniEX: An Effective and Efficient Framework for Unified Information Extraction via a Span-extractive Perspective
Yang Ping (International Digital Economy Academy), Jiaxing Zhang (International Digital Economy Academy)
ClassificationComputational EfficiencyRepresentation LearningData-Centric LearningTransformerLarge Language ModelPrompt EngineeringAuto EncoderTextBenchmark
🎯 What it does: Propose a unified information extraction framework called UniEX, which transforms all IE tasks into a joint problem of span detection, classification, and association, achieving efficient extraction through trilinear attention.
Unified Demonstration Retriever for In-Context Learning
Xiaonan Li (Fudan University), Xipeng Qiu (Fudan University)
RetrievalTransformerTextBiomedical DataFinance Related
🎯 What it does: Propose a Unified Demonstration Retriever (UDR) that can retrieve high-quality demonstrations for a large number of different NLP tasks by using list ranking based on language model feedback and iterative candidate mining on a multi-task bi-encoder.
Unifying Cross-Lingual and Cross-Modal Modeling Towards Weakly Supervised Multilingual Vision-Language Pre-training
Zejun Li (Fudan University), Zhongyu Wei (Fudan University)
RetrievalRepresentation LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a weakly supervised multilingual vision-language pre-training framework that unifies cross-lingual and cross-modal modeling using only English image-text pairs and multilingual text corpora.
UniLG: A Unified Structure-aware Framework for Lyrics Generation
Tao Qian (Renmin University of China), Qin Jin (Renmin University of China)
GenerationTransformerLarge Language ModelTextMultimodalityAudio
🎯 What it does: Proposes a unified structure-aware lyric generation framework, UniLG, which utilizes composite templates incorporating both textual and musical information to achieve multi-condition generation;
UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization
Yulong Chen (Zhejiang University), Yue Zhang (Westlake University)
GenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Proposed a unified few-shot text summarization framework called UNISUMM, and released the SUMMZOO benchmark containing 8 diverse tasks; achieved efficient adaptation to any few-shot summarization task through multi-task pre-training and prefix tuning.
UniTRec: A Unified Text-to-Text Transformer and Joint Contrastive Learning Framework for Text-based Recommendation
Zhiming Mao (Chinese University of Hong Kong), Kam-Fai Wong (Chinese University of Hong Kong)
Recommendation SystemTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Implemented a unified text-to-text Transformer framework named UniTRec, which uses local and global attention for dual-layer encoding of multi-round user history, and combines a pre-trained decoder to calculate the perplexity of candidate text as a matching signal to complete the text recommendation task.
UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units
Hirofumi Inaguma (FAIR, Meta AI), Juan Pino (FAIR, Meta AI)
Convolutional Neural NetworkTransformerTextAudio
🎯 What it does: Proposed and implemented UnitY, a two-step direct speech-to-speech translation model that first generates text representations through subword decoding, then generates target speech via discrete acoustic unit decoding.
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor
Or Honovich (Tel Aviv University), Timo Schick (Meta AI)
Data SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Use pre-trained language models (e.g., text-davinci-002) to automatically generate approximately 240k instructional samples (including inputs and outputs) through a few human-annotated examples for instruction tuning.
Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars
Songlin Yang (ShanghaiTech University), Yoon Kim (MIT)
Computational EfficiencyRepresentation LearningText
🎯 What it does: Investigated unsupervised discontinuous constituent analysis using mildly context-sensitive grammar, adopting a probabilistic linear context-free rewriting system (LCFRS) formulation, fixing the rule structure in advance, and focusing on maximum likelihood parameter learning.