EMNLP 2023 Papers — Page 8
Conference on Empirical Methods in Natural Language Processing · 1047 papers
ORCHID: A Chinese Debate Corpus for Target-Independent Stance Detection and Argumentative Dialogue Summarization
Xiutian Zhao (Huawei IT Innovation and Research Center), Wei Peng (Huawei IT Innovation and Research Center)
ClassificationGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed the first Chinese target-agnostic stance detection and debate summarization dataset, ORCHID, containing 1,218 debates, 14,133 fully annotated speeches, and 2,436 stance-specific summaries, and conducted benchmark experiments on this dataset; proposed the task of stance-specific summary integration; combined spoken debate corpus with multi-domain and long-text characteristics;
OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding
Zhan Shi (Bytedance), Xiaodan Zhu (Queen's University)
Representation LearningTransformerContrastive LearningText
🎯 What it does: Research and overcome semantic representation distortion in unsupervised sentence embeddings caused by surface structure deviations, proposing the OssCSE model and systematically evaluating its performance.
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future
Linyi Yang (Westlake University), Yue Zhang (Westlake University)
Domain AdaptationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextReview/Survey PaperBenchmark
🎯 What it does: A systematic review of 'discrete out-of-distribution (OOD) generalization' in natural language processing, categorizing problems into two major classes based on data and features, and providing multi-dimensional methods and evaluation frameworks, with particular focus on the OOG challenges of large-scale language models (LLM).
Outlier Dimensions Encode Task Specific Knowledge
William Rudman (Brown University), Carsten Eickhoff (Brown University)
ClassificationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigate the persistence of outlier dimensions in large language models and whether they retain task-specific knowledge after fine-tuning, exploring whether a single dimension can accomplish downstream binary classification tasks.
Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling
Xiuying Wei (Beihang University), Xianglong Liu (Beihang University)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper addresses the outlier problem in the activation of Transformer language models by proposing the Outlier Suppression+ (OS+) framework, which achieves equivalent quantization through channel-level shift and scaling operations, and ensures model equivalence between FP and quantized versions by leveraging a migration pattern.
P5: Plug-and-Play Persona Prompting for Personalized Response Selection
Joosung Lee (NAVER), Donghun Lee (NAVER)
RetrievalTransformerPrompt EngineeringContrastive LearningText
🎯 What it does: Proposed the P5 method, which achieves personalized response selection under zero-shot and fine-tuning scenarios by inputting the persona sentence with the highest similarity to the response as a prompt along with the dialogue context;
PAC-tuning: Fine-tuning Pre-trained Language Models with PAC-driven Perturbed Gradient Descent
Guangliang Liu (Michigan State University), Rongrong Wang (Michigan State University)
ClassificationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose a two-stage fine-tuning method called PAC-tuning, which learns noise variance through PAC-Bayes training and subsequently perturbs gradient descent with this noise, enhancing the generalization ability of pre-trained language models on few-shot text classification tasks.
PALS: Personalized Active Learning for Subjective Tasks in NLP
Kamil Kanclerz (Wrocław University of Science and Technology), Przemyslaw Kazienko (Wrocław University of Science and Technology)
ClassificationTransformerText
🎯 What it does: Propose the PALS (Personalized Active Learning for Subjective NLP Tasks) framework, which selects suitable annotation samples for each user using five active learning methods based on controversy and individual preference metrics, thereby enhancing the performance of personalized models.
Parameter-Efficient Language Model Tuning with Active Learning in Low-Resource Settings
Josip Jukić (University of Zagreb), Jan Snajder (University of Zagreb)
ClassificationTransformerSupervised Fine-TuningText
🎯 What it does: This paper studies the combination of parameter-efficient fine-tuning (PEFT) and active learning (AL) in low-resource text classification tasks, assessing their effectiveness in reducing annotation costs and improving performance.
Parameter-efficient Tuning for Large Language Model without Calculating Its Gradients
Feihu Jin (Institute of Automation Chinese Academy of Sciences), Chengqing Zong (Institute of Automation Chinese Academy of Sciences)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Leverage a gradient-agnostic bridging model to transfer parameter-efficient tuning modules learned on a small language model to a large language model, directly using them during inference without needing to update the large model's parameters.
Paraphrase Types for Generation and Detection
Jan Philip Wahle (University of Göttingen), Terry Ruas (University of Göttingen)
ClassificationGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Proposes two task types based on fine-grained rewriting categories—rewriting type generation and rewriting type detection, emphasizing the consideration of specific linguistic perturbations in both generation and detection;
PEFTDebias : Capturing debiasing information using PEFTs
Sumit Agarwal (Carnegie Mellon University), Srijan Bansal (Carnegie Mellon University)
Explainability and InterpretabilityData-Centric LearningTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose a debiasing method called PEFTDebias based on parameter-efficient fine-tuning (PEFT), which first trains the PEFT module on unlabeled axis-related corpora using CDA, and then freezes this module during downstream tasks for fine-tuning to maintain debiasing effects;
Penalty Decoding: Well Suppress the Self-Reinforcement Effect in Open-Ended Text Generation
Wenhong Zhu (Shanghai Jiao Tong University), Rui Wang (Shanghai Jiao Tong University)
GenerationTransformerLarge Language ModelText
🎯 What it does: Proposed and implemented the 'penalty decoding' method, utilizing three techniques—repetition penalty, forgetting mechanism, and length penalty—to suppress self-reinforcement phenomena during the generation process, thereby enhancing the quality of open-ended text generation.
People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detection
Indira Sen (RWTH Aachen University), Claudia Wagner (RWTH Aachen University)
ClassificationData SynthesisAnomaly DetectionTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Compare the effectiveness of manually generated versus automatically generated Counterfactually Augmented Data (CAD) in gender bias and hate speech detection tasks.
Personalized Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation
Hailin Chen (Nanyang Technological University), Shafiq Joty (Nanyang Technological University)
Knowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: Proposes a Personalised Distillation method, where the student model first generates code and executes it to obtain error feedback, then the teacher (ChatGPT) provides adaptive improvements based on the student's errors, followed by supervised fine-tuning of the student model; this process achieves an interactive, customized learning workflow; significant improvements are achieved in code generation tasks.
PHD: Pixel-Based Language Modeling of Historical Documents
Nadav Borenstein (University of Copenhagen), Isabelle Augenstein (University of Copenhagen)
RestorationData SynthesisTransformerSupervised Fine-TuningVision Language ModelImageText
🎯 What it does: Developed an OCR-free pixel-level historical document language model, PHD, trained to reconstruct occluded image patches and applied to historical question answering tasks.
Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors
Nikita Mehandru (University of California, Berkeley), Niloufar Salehi (University of California, Berkeley)
Large Language ModelTextBiomedical DataElectronic Health Records
🎯 What it does: This paper evaluates the effectiveness of Quality Estimation (QE) and Back-Translation (BT) as feedback mechanisms in helping physicians determine whether machine translation (MT) outputs are suitable for patients to read, designing and conducting a randomized controlled experiment.
PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training
Yunyi Zhang (University of Illinois Urbana-Champaign), Jiawei Han (University of Illinois Urbana-Champaign)
ClassificationTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose the PIEClass method, which generates pseudo labels using zero-shot prompts and completes weakly supervised text classification through noise-robust iterative ensemble training.
PK-ICR: Persona-Knowledge Interactive Multi-Context Retrieval for Grounded Dialogue
Minsik Oh, Guoyin Wang
RetrievalTransformerSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: For multi-contextual dialogue systems, the Persona-Knowledge Dual Context Retrieval (PK-ICR) method is proposed, leveraging QA cross-task prompting to achieve zero-shot knowledge retrieval and fine-grained persona scoring, and introducing the Null-Positive Rank Test to evaluate the ranking performance of hard negative samples.
Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts
Tengxiao Liu (Fudan University), Zheng Zhang (Amazon AWS AI)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposes the XoT framework, which enables large language models to dynamically switch between multiple thinking modes such as CoT, PoT, and EoT in mathematical reasoning through three modules: planning, reasoning, and verification, aiming for more accurate answers.
POE: Process of Elimination for Multiple Choice Reasoning
Chenkai Ma (University of Electronic Science and Technology of China), Xinya Du (University of Texas at Dallas)
ClassificationLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Propose Process of Elimination (POE), a two-step scoring method: first score each option and eliminate incorrect ones based on scores, then rescore the remaining options and make a final judgment.
Pointwise Mutual Information Based Metric and Decoding Strategy for Faithful Generation in Document Grounded Dialogs
Yatin Nandwani (IBM Research, AI), Luis Lastras (IBM Research, AI)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed a novel conditional point-wise mutual information-based metric PMI-FAITH and the corresponding decoding strategy PMI-DECODE for evaluating and generating dialogue responses consistent with documents.
Poisoning Retrieval Corpora by Injecting Adversarial Passages
Zexuan Zhong (Princeton University), Danqi Chen (Princeton University)
RetrievalRepresentation LearningAdversarial AttackTransformerContrastive LearningText
🎯 What it does: Propose a corpus poisoning attack targeting dense retrieval systems, generating adversarial passages and injecting them into the retrieval corpus to mislead the model into returning these passages
Polar Ducks and Where to Find Them: Enhancing Entity Linking with Duck Typing and Polar Box Embeddings
Mattia Atzeni (Meta AI), Nicola Cancedda (Meta AI)
RetrievalRepresentation LearningTransformerLarge Language ModelContrastive LearningTextGraph
🎯 What it does: This paper proposes the DUCK method, which improves dense retrieval models for entity linking by leveraging relationship information from knowledge graphs through 'duck typing' type definitions and polar coordinate box embeddings, enabling entity embedding spaces to cluster by type.
Polyglot or Not? Measuring Multilingual Encyclopedic Knowledge in Foundation Models
Tim Schott (University of California, Berkeley), Shreshta Bhat (University of California, Berkeley)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextBenchmark
🎯 What it does: Evaluate the ability of base models to recall encyclopedic facts in multilingual environments, constructing a dataset with 20 languages, 303k facts and adversarial hypotheses, and comparing five models in multilingual and monolingual (English) tests.
Post-hoc Utterance Refining Method by Entity Mining for Faithful Knowledge Grounded Conversations
Yoonna Jang (Korea University), Heuiseok Lim (Korea University)
GenerationRetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Propose a post-refinement method called REM, which improves the authenticity and consistency of generated statements in knowledge-driven dialogues by mining entities and implicitly learning entity information from given knowledge.
Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions
Kazuki Irie (Harvard University), Jürgen Schmidhuber (Swiss AI Lab IDSIA, USI & SUPSI)
RecognitionTransformerTextSequential
🎯 What it does: Investigate the computational power of linear Transformer (LT) and its Fast Weight Programmer (FWP), and conduct experimental validation on formal language recognition tasks.
Pragmatic Reasoning Unlocks Quantifier Semantics for Foundation Models
Yiyuan Li (UNC Chapel Hill), Shashank Srivastava (UNC Chapel Hill)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes the PRESQUE framework, which combines natural language reasoning with pragmatic reasoning (RSA) to infer the percentage range of quantifiers using pre-trained language models.
PRCA: Fitting Black-Box Large Language Models for Retrieval Question Answering via Pluggable Reward-Driven Contextual Adapter
Haoyan Yang (Ping An Technology Co., Ltd.), Jing Xiao (Ping An Technology Co., Ltd.)
RetrievalKnowledge DistillationTransformerLarge Language ModelReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: Propose a pluggable reward-driven context adapter (PRCA) in a retrieval-augmented QA framework, treating large language models (LLMs) as black-box generators. PRCA performs information distillation on retrieved documents, enabling the generator to obtain higher-quality context while remaining frozen, thereby improving QA quality.
Pre-Trained Language Models Augmented with Synthetic Scanpaths for Natural Language Understanding
Shuwen Deng (University of Potsdam), Lena Jäger (University of Zurich)
ClassificationData SynthesisRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: By coupling a synthetic scan path (eye movement trajectory) generator with pre-trained language models, the performance of natural language processing tasks can be improved without requiring real eye-tracking data.
Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification
Mujeen Sung (Korea University), Vittorio Castelli (AWS AI Labs)
ClassificationDomain AdaptationMeta LearningTransformerContrastive LearningText
🎯 What it does: This paper proposes a pre-training method that utilizes intent role labeling (IRL) to generate pseudo intent names and perform intent-aware contrastive learning, training an encoder named PIE that performs excellently in zero-shot and few-shot intent classification tasks.
Pre-training Language Models for Comparative Reasoning
Mengxia Yu (University Of Notre Dame), Meng Jiang (University Of Notre Dame)
Data SynthesisRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextGraphBenchmark
🎯 What it does: Propose a pre-training framework for comparative reasoning that automatically collects entity comparison pairs using structured knowledge graphs and unstructured text, and enhances the model's comparative reasoning ability through three text-to-text pre-training tasks.
Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration
Yiquan Wu (Zhejiang University), Kun Kuang (Zhejiang University)
ClassificationConvolutional Neural NetworkTransformerLarge Language ModelContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes a precedent-enhanced legal judgment prediction framework called PLJP, which leverages a domain model to provide candidate labels and precedent retrieval, followed by an LLM synthesizing precedents in context to make the final judgment.
Predict and Use: Harnessing Predicted Gaze to Improve Multimodal Sarcasm Detection
Divyank Tiwari, Pushpak Bhattacharyya (IIT Bombay)
ClassificationConvolutional Neural NetworkTransformerVideoTextMultimodalityAudio
🎯 What it does: In a multimodal dialogue context, utilizing collected eye movement data and predicted synthetic eye movement features to enhance sarcasm detection performance.
Predict the Future from the Past? On the Temporal Data Distribution Shift in Financial Sentiment Classifications
Yue Guo (Hong Kong University of Science and Technology), Yi Yang (Hong Kong University of Science and Technology)
ClassificationTransformerSupervised Fine-TuningTextTime SeriesFinance Related
🎯 What it does: This paper systematically investigates the robustness of financial sentiment classification under time distribution shift, and proposes a two-stage method that combines OOD detection with autoregressive time series models to alleviate performance degradation on future data.
Predictive Chemistry Augmented with Text Retrieval
Yujie Qian (MIT), Regina Barzilay (MIT)
Drug DiscoveryTransformerContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Propose the TextReact method, which integrates retrieved natural language text with chemical reaction data to enhance the performance of chemical prediction models.
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction
Ji Qi (Tsinghua University), Xu Bin (Tsinghua University)
TextBenchmark
🎯 What it does: Proposed the ROBUST benchmark to evaluate the robustness of open information extraction models under syntactic and expression distribution drift;
Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language Models
Aly Kassem (University of Windsor), Sherif Saad (Deakin University)
Safty and PrivacyTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Train language models to generate paraphrases with low similarity to training data, achieving 'forgetting' of sensitive information through reinforcement learning with negative similarity rewards.
PRESTO: A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs
Rahul Goel (Google Inc), Zhou Yu (Google Inc)
Data-Centric LearningTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: Built and released a multilingual, task-oriented dialogue parsing dataset named PRESTO, which includes real interaction phenomena, and provided baseline model results.
PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering
Wookje Han (Columbia University), Kyungjae Lee (LG AI Research)
TransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Propose the PreWoMe method, which treats presuppositions in questions as working memory, generating feedback and actions through intermediate steps to uniformly handle long-form question-answering tasks with ambiguity or incorrect assumptions without requiring model parameter updates.
Primacy Effect of ChatGPT
Yiwei Wang (University of California Los Angeles), Bryan Hooi (National University Of Singapore)
ClassificationTransformerLarge Language ModelText
🎯 What it does: This paper introduces randomness in label order within ChatGPT's judgment-based natural language understanding tasks, investigating and confirming the existence of a primacy effect in ChatGPT's predictions.
Privacy Implications of Retrieval-Based Language Models
Yangsibo Huang (Princeton University), Danqi Chen (Princeton University)
Safty and PrivacyTextRetrieval-Augmented Generation
🎯 What it does: Investigate the privacy leakage risks of retrieval-based language models (kNN-LM) and propose mitigation strategies for both targeted and non-targeted leaks
Program Translation via Code Distillation
Yufan Huang (Microsoft Cloud and AI), Neel Sundaresan (Microsoft Cloud and AI)
AI Code AssistantTransformerLarge Language ModelContrastive LearningTextGraph
🎯 What it does: This paper proposes a program translation framework that utilizes distilled code as an intermediary for cross-language translation, and designs corresponding distilled code compilers and multi-language distilled code decompilers, achieving high-quality code translation at the function level under unsupervised pre-training.
Promoting Topic Coherence and Inter-Document Consorts in Multi-Document Summarization via Simplicial Complex and Sheaf Graph
Yash Kumar Atri (IIIT-Delhi), Vikram Goyal (IIIT-Delhi)
GenerationTransformerText
🎯 What it does: Propose a BART-based multi-document summarization model called FABRIC, which enhances cross-document coherence and factual consistency by leveraging theme-assisted document segmentation, degenerate composite layers, and layered graph attention.
Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models
Shuai Zhao (Jinan University), Jie Fu (Hong Kong University of Science and Technology)
ClassificationAdversarial AttackTransformerSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: ProAttack utilizes the Prompt itself as a trigger to perform clean-label text backdoor attacks, injecting backdoors into the model without altering the original labels.
Prompt-based Logical Semantics Enhancement for Implicit Discourse Relation Recognition
Chenxu Wang (Beijing Institute of Technology), Mu Huang (Beijing Institute of Technology)
RecognitionRepresentation LearningTransformerPrompt EngineeringContrastive LearningText
🎯 What it does: Designed a Prompt-based Logical Semantics Enhancement (PLSE) method, which utilizes unannotated explicit connective data for Cloze-Prompt connective prediction during pre-training, and introduces mutual information (MI) maximization learning in downstream implicit discourse relation recognition tasks to capture global logical semantics, thereby improving the model's performance in identifying implicit relations.
Prompt-Based Monte-Carlo Tree Search for Goal-oriented Dialogue Policy Planning
Xiao Yu (Columbia University), Zhou Yu (Columbia University)
OptimizationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose GDP-ZERO, an open MCTS decision framework that requires no model training, leveraging large language models to simulate user-system interactions and evaluate task progress in dialogue tree search, directly planning goal-oriented dialogue strategies;
Prompting is not a substitute for probability measurements in large language models
Jennifer Hu (Harvard University), Roger Levy (Massachusetts Institute of Technology)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Compare the effectiveness of prompting and direct probability distribution reading in evaluating language knowledge in large language models.
Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation
Yuanyuan Liang (East China Normal University), Yunshi Lan (East China Normal University)
GenerationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Propose the KQG-CoT framework, which selects representative logical forms by structural encoding and clustering from an unlabeled data pool, then constructs prompts using chain-of-thought (CoT) to enable large language models to generate natural language questions consistent with given logical forms under few-shot settings.
Prompting Scientific Names for Zero-Shot Species Recognition
Shubham Parashar (Texas A&M University), Shu Kong (Institute of Collaborative Innovation)
ClassificationRecognitionLarge Language ModelPrompt EngineeringVision Language ModelImage
🎯 What it does: This paper studies zero-shot species recognition using pre-trained vision-language models (OpenCLIP) and proposes translating Latin/Greek scientific names into common English names to construct more effective text prompts.
Prompting with Pseudo-Code Instructions
Mayank Mishra (IBM Research AI), Srikanth Tamilselvam (IBM Research AI)
ClassificationGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose and evaluate the use of pseudo-code as a prompt method, investigating its impact on large language models (LLMs) across multi-task scenarios (classification, question answering, and generation);
PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation
Gaurav Sahu (University of Waterloo), Issam Laradji
ClassificationData SynthesisKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose PromptMix, a two-step prompting method that first generates mixed augmented samples near class boundaries using an LLM, and then re-labels the generated samples with another LLM to obtain high-quality augmented data; subsequently, these data are used to train small models such as DistilBERT/BERT.
PromptST: Abstract Prompt Learning for End-to-End Speech Translation
Tengfei Yu (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
TransformerSupervised Fine-TuningPrompt EngineeringBenchmarkAudio
🎯 What it does: This paper introduces soft prompts into end-to-end speech translation models to enhance the representation capability of high-level encoders.
PROSE: A Pronoun Omission Solution for Chinese-English Spoken Language Translation
Ke Wang (Huawei IT Innovation and Research Center), Wei Peng (Huawei IT Innovation and Research Center)
TransformerContrastive LearningTextBenchmark
🎯 What it does: Built PROSE, a Chinese-English spoken translation document-level dataset covering four spoken genres (conversations, TV series, movies, vlogs), and analyzed and addressed translation errors caused by omitted pronouns in Chinese.
Prototype-based HyperAdapter for Sample-Efficient Multi-task Tuning
Hao Zhao (Beijing University of Posts and Telecommunications), Zhaofeng He (Beijing University of Posts and Telecommunications)
Representation LearningMeta LearningSupervised Fine-TuningContrastive LearningText
🎯 What it does: Propose Prototype-based HyperAdapter (PHA), which generates task-specific adapters in multi-task and few-shot transfer scenarios through an instance-dense retriever and prototypical hypernetwork.
PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer
Lichang Chen (University of Maryland), Minhao Cheng (Hong Kong University of Science and Technology)
Representation LearningData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose Prompt Tuning with Perturbation-based Regularizer (PTP), which smooths the loss landscape by introducing random noise or adversarial perturbations in the text and embedding spaces, addressing training instability in prompt tuning and enhancing performance.
Pushdown Layers: Encoding Recursive Structure in Transformer Language Models
Shikhar Murty (Stanford University), Christopher Manning (Stanford University)
Representation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose Pushdown Layers that integrate stack memory with Transformer self-attention to automatically infer and maintain recursive syntactic structures.
q2d: Turning Questions into Dialogs to Teach Models How to Search
Yonatan Bitton (Hebrew University of Jerusalem), Enav Weinreb (Google Research)
Data SynthesisRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Using large language models to automatically generate information retrieval dialogues from questions, and constructing a query generation dataset through rigorous filtering
QA-NatVer: Question Answering for Natural Logic-based Fact Verification
Rami Aly (University of Cambridge), Andreas Vlachos (University of Cambridge)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose a fact verification system QA-NatVer based on natural logic, which transforms the prediction of natural logic operators into a question-answering task to achieve few-shot, interpretable fact verification.
QTSumm: Query-Focused Summarization over Tabular Data
Yilun Zhao (Yale University), Arman Cohan (Yale University)
GenerationTransformerLarge Language ModelTabularBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Studied the query-focused table summarization task, constructed the QTSUMM benchmark dataset, and proposed the REFACTOR method to retrieve and reason about table information to generate fact-enhanced summaries.
Quantifying Character Similarity with Vision Transformers
Xinmei Yang (Harvard University), Melissa Dell (Harvard University)
RecognitionRetrievalRepresentation LearningTransformerContrastive LearningText
🎯 What it does: Built a self-supervised vision transformer model (HOMOGLYPH) to measure the visual similarity between homoglyphs in OCR text, using this similarity as the character substitution cost in the Levenshtein edit distance, thereby improving record linkage accuracy across different OCR engines, scripts, and languages (CJK and ancient scripts).
Quantifying the redundancy between prosody and text
Lukas Wolf (ETH Zürich), Tamar I. Regev (MIT)
TransformerLarge Language ModelSupervised Fine-TuningTextAudio
🎯 What it does: This paper uses large language models to estimate the redundancy between speech prosody features and textual information, quantifying their mutual information;
QUDeval: The Evaluation of Questions Under Discussion Discourse Parsing
Yating Wu (University of Texas at Austin), Junyi Jessy Li (University of Texas at Austin)
GenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposes the QUDEVAL evaluation framework and dataset for fine-grained assessment of the quality of question-under-discussion (QUD) generation in dialogue parsing.
Query Rewriting in Retrieval-Augmented Large Language Models
Xinbei Ma (Shanghai Jiao Tong University), Nan Duan (Microsoft Research Asia)
RetrievalComputational EfficiencyTransformerLarge Language ModelReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: Proposed the Rewrite-Retrieve-Read framework, introducing a query rewriting step into retrieval-augmented LLMs and training a tunable lightweight rewriter to enhance retrieval quality and final answer accuracy.
Query-as-context Pre-training for Dense Passage Retrieval
Xing W (Institute of Information Engineering, Chinese Academy of Sciences), Songlin Hu (Institute of Information Engineering, Chinese Academy of Sciences)
RetrievalRepresentation LearningTransformerAuto EncoderContrastive LearningText
🎯 What it does: Proposes a pre-training method based on using queries as context to enhance the effectiveness of dense passage retrieval.
Query2doc: Query Expansion with Large Language Models
Liang Wang (Microsoft Research), Furu Wei (Microsoft Research)
RetrievalLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a query expansion method called query2doc based on large language models, which supplements the original query with generated pseudo-documents during the retrieval phase;
Question Answering as Programming for Solving Time-Sensitive Questions
Xinyu Zhu (Tsinghua University), Yujiu Yang (Tsinghua University)
AI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose converting the QA task into programming, using LLMs to parse the problem and document content into Python dictionaries/lists, then verifying and time-matching the extracted facts through two custom steps, Check and Match, to obtain answers that meet temporal constraints.
R2H: Building Multimodal Navigation Helpers that Respond to Help Requests
Yue Fan (University of California, Santa Cruz), Xin Wang
Autonomous DrivingReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: Proposed the Respond to Help Requests (R2H) benchmark to evaluate the response capability of multimodal navigation assistant agents in dialogue history and interactive processes, and developed two assistant models based on this benchmark: SeeRee (self-supervised vision-language Transformer combined with COS Attention Mask and Parse by Step preprocessing) and zero-shot multimodal LLM (mPLUG-Owl).
RainProof: An Umbrella to Shield Text Generator from Out-Of-Distribution Data
Maxime Darrin (MILA - Quebec AI Institute), Pierre Colombo (Paris-Saclay University)
GenerationDomain AdaptationAnomaly DetectionTextBenchmark
🎯 What it does: This paper proposes a black-box OOD detection framework for text generation models called RAINPROOF, and constructs the LOFTER benchmark tailored to language, domain, and dialogue shift, to evaluate model robustness in open-world scenarios.
Random Entity Quantization for Parameter-Efficient Compositional Knowledge Graph Representation
Jiaang Li (University of Science and Technology of China), Zhendong Mao (University of Science and Technology of China)
Computational EfficiencyRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: This paper studies the entity quantization problem in knowledge graph representation learning, proposing a random quantization method for entities and proving that it can achieve the same or better performance compared to complex quantization strategies in parameter-efficient compositional knowledge graph representation methods;
RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction
Shiao Meng (Tsinghua University), Lijie Wen (Tsinghua University)
Representation LearningMeta LearningTransformerContrastive LearningText
🎯 What it does: Propose a relation-aware prototype learning framework, RAPL, for few-sample document-level relation extraction.
Rather a Nurse than a Physician - Contrastive Explanations under Investigation
Oliver Eberle (Technische Universität Berlin), Stephanie Brandl (University of Copenhagen)
Explainability and InterpretabilityTransformerTextBiomedical Data
🎯 What it does: Compared controlled and uncontrolled explanations between humans and models in text classification tasks to examine whether controlled explanations better align with human cognition.
Rationale-Enhanced Language Models are Better Continual Relation Learners
Weimin Xiong (Peking University), Sujian Li (Peking University)
TransformerLarge Language ModelPrompt EngineeringContrastive LearningText
🎯 What it does: Propose the RationaleCL framework, introducing rationales generated by large language models (LLMs) into continual relation extraction (CRE), enhancing the model's ability to distinguish similar relations through multi-task reasoning optimization and contrastive reasoning replay.
Re^3Dial: Retrieve, Reorganize and Rescale Conversations for Long-Turn Open-Domain Dialogue Pre-training
Jiaxin Wen (Tsinghua University), Minlie Huang (Tencent Inc)
Data SynthesisTransformerLarge Language ModelContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Automatically construct a billion-level long-turn dialogue corpus by retrieving, recombining, and remarking short-turn dialogues to concatenate them into long turns, thereby enhancing the utilization of long contexts;
Reader: Model-based language-instructed reinforcement learning
Nicola Dainese (Aalto University), Alexander Ilin (Aalto University)
Explainability and InterpretabilityTransformerReinforcement LearningAuto EncoderWorld ModelText
🎯 What it does: In this paper, the authors propose a language-instruction-based model-driven reinforcement learning framework called Reader, which utilizes an interpretable world model and Monte Carlo Tree Search (MCTS) to complete tasks in the Read To Fight Monsters (RTFM) environment.
Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms
Seungju Han (Seoul National University), Youngjae Yu (Yonsei University)
ClassificationExplainability and InterpretabilityKnowledge DistillationLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Propose and construct a new multimodal benchmark dataset called NORMLENS to evaluate models' reasoning ability about defeasible commonsense norms in visual-semantic contexts; two tasks are defined on this dataset: providing moral judgments for given image-action pairs and generating corresponding explanations.
Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction
Chong Zhang (Fudan University), Tao Gui (Fudan University)
ClassificationTransformerVision Language ModelImageTextBenchmark
🎯 What it does: This paper proposes the Token Path Prediction (TPP) framework, modeling the named entity recognition (NER) problem in visually rich documents as predicting token paths in a complete directed graph, thereby addressing the issue of BIO label failure caused by the uncertainty of text reading order generated by OCR.
Reasoning with Language Model is Planning with World Model
Shibo Hao (University Of San Diego), Zhiting Hu (University Of San Diego)
TransformerLarge Language ModelWorld ModelText
🎯 What it does: Proposed a framework called RAP that enables large language models to simulate world models during reasoning and plan through Monte Carlo Tree Search (MCTS), thereby enhancing reasoning quality.
ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph
Jinhao Jiang (Renmin University of China), Ji-Rong Wen (Alibaba Group)
Data SynthesisComputational EfficiencyGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningGraphBenchmark
🎯 What it does: Constructed a ReasoningLM capable of performing structural subgraph reasoning within a single pre-trained language model;
ReCEval: Evaluating Reasoning Chains via Correctness and Informativeness
Archiki Prasad (UNC Chapel Hill), Mohit Bansal (UNC Chapel Hill)
TransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Proposes the RECEVAL framework, which utilizes reference-free NLI and V-Information metrics to conduct fine-grained evaluation of the correctness and informativeness of multi-step reasoning chains.
Reconstruct Before Summarize: An Efficient Two-Step Framework for Condensing and Summarizing Meeting Transcripts
Haochen Tan (City University of Hong Kong), Linqi Song (City University of Hong Kong)
CompressionRepresentation LearningTransformerAuto EncoderText
🎯 What it does: Propose a two-step framework RbS, first identifying key information through self-supervised reconstruction, then generating summaries by compressing text using relative position bucketization.
Recurrent Neural Language Models as Probabilistic Finite-state Automata
Anej Svete (ETH Zurich), Ryan Cotterell (ETH Zurich)
Computational EfficiencyRepresentation LearningRecurrent Neural NetworkText
🎯 What it does: This paper investigates the expressive power of Elman RNN language models (specifically HRNN with Heaviside activation function) in probabilistic finite state automata, proving that HRNN can represent all probability distributions equivalent to deterministic probabilistic finite state automata (DPFSAs), and providing theoretical lower bounds on the required hidden layer size for RNNs to simulate PFSA.
Reduce Human Labor On Evaluating Conversational Information Retrieval System: A Human-Machine Collaboration Approach
Chen Huang (Sichuan University), Jiancheng Lv (Sichuan University)
RetrievalTransformerAgentic AIText
🎯 What it does: Proposed a human-machine collaborative evaluation framework, HumCoE, to significantly reduce manual annotation costs in evaluating conversational information retrieval (CIR) systems while maintaining evaluation accuracy.
Reducing Sequence Length by Predicting Edit Spans with Large Language Models
Masahiro Kaneko (MBZUAI), Naoaki Okazaki (Tokyo Institute of Technology)
GenerationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose using edit spans to predict modified parts in the source text, thereby reducing the target sequence length and lowering inference costs in local sequence transduction tasks.
Referring Image Segmentation via Joint Mask Contextual Embedding Learning and Progressive Alignment Network
Ziling Huang (University of Tokyo), Shin’ichi Satoh
SegmentationConvolutional Neural NetworkVision Language ModelMultimodality
🎯 What it does: Propose a joint mask and context embedding learning network (JMCELN), achieving multi-stage segmentation reasoning through dynamically learned context embeddings and a progressive alignment network;
Reformulating NLP tasks to Capture Longitudinal Manifestation of Language Disorders in People with Dementia.
Dimitris Gkoumas (Queen Mary University of London), Maria Liakata (Queen Mary University of London)
ClassificationRecognitionExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBiomedical DataAlzheimer's Disease
🎯 What it does: Leveraging the medium-scale pre-trained language model RoBERTa, transforming language impairment patterns in narrative speech transcripts from dementia patients and healthy subjects into multiple NLP tasks (text-to-text generation, natural language inference, prompt learning, etc.), generating interpretable digital communication and impairment markers through model probabilities, and evaluating their changes and associations with clinical indicators on longitudinal samples.
Regulation and NLP (RegNLP): Taming Large Language Models
Catalina Goanta (Utrecht University), Gerasimos Spanakis (Maastricht University)
Safty and PrivacyLarge Language ModelText
🎯 What it does: This paper proposes the RegNLP (Regulation and NLP) research direction, aiming to systematically integrate regulatory studies with natural language processing to promote risk assessment and governance of large language models (LLMs).
Reinforced Target-driven Conversational Promotion
Huy Dao, Yuxiang Nie (Hong Kong University of Science and Technology)
Recommendation SystemTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: This paper proposes an RTCP, a goal-driven conversation promotion framework that integrates short-term and long-term planning along with prefix tuning to generate high-quality dialogues for targeted products;
Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning
Swaroop Nath (IIT Bombay), Harshad Khadilkar (TCS Research)
GenerationTransformerSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Developed a query-focused summary generation model based on reinforcement learning, resolving the conflict between teacher forcing and RL in Transformers, and proposed a semantic similarity reward based on the cluster hypothesis.
Relation-aware Ensemble Learning for Knowledge Graph Embedding
Ling Yue (Tsinghua University), Yefeng Zheng (Tencent)
OptimizationRepresentation LearningHyperparameter SearchGraph
🎯 What it does: This paper proposes a relation-aware ensemble learning method called RelEns-DSC for knowledge graph embedding.
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Fengji Zhang (City University of Hong Kong), Weizhu Chen (Microsoft Corporation)
AI Code AssistantTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper proposes the RepoCoder framework, achieving repository-level code completion by retrieving and utilizing code snippets from other files to enhance context;
Representative Demonstration Selection for In-Context Learning with Two-Stage Determinantal Point Process
Zhao Yang (University of Chinese Academy of Sciences), Kang Liu (University of Chinese Academy of Sciences)
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a two-stage Determinantal Point Process (DPP) method to select representative examples from the training set for In-Context Learning (ICL).
ReSee: Responding through Seeing Fine-grained Visual Knowledge in Open-domain Dialogue
Haoqin Tu, Zhongliang Yang (Huawei Noah's Ark Lab)
RetrievalTransformerLarge Language ModelVision Language ModelImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Constructed multimodal dialogue datasets RESEEWoW and RESEEDD with fine-grained visual knowledge, and proposed the RESEE framework which significantly improves generation quality by integrating visual information into traditional dialogue models.
ReTAG: Reasoning Aware Table to Analytic Text Generation
Deepanway Ghosal (Singapore University of Technology and Design), Aravindan Raghuveer (Google Research)
GenerationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextTabularChain-of-Thought
🎯 What it does: Proposes the RETAG model, which enables controllable generation of multiple reasoning types in table summarization, balancing both descriptive and analytical summaries;
Rethinking and Improving Multi-task Learning for End-to-end Speech Translation
Yuhao Zhang (Northeastern University), Jingbo Zhu (Northeastern University)
Knowledge DistillationTransformerContrastive LearningTextAudio
🎯 What it does: By analyzing gradient consistency in multi-task learning, an improved multi-task learning framework (IMTL) is proposed, achieving state-of-the-art (SOTA) performance in end-to-end speech translation without fine-tuning.
Rethinking Model Selection and Decoding for Keyphrase Generation with Pre-trained Sequence-to-Sequence Models
Di Wu (University of California Los Angeles), Kai-Wei Chang (University of California Los Angeles)
GenerationTransformerLarge Language ModelText
🎯 What it does: This paper systematically studies the impact of model scale, domain pre-training, task adaptation, and decoding strategies on keyphrase generation (KPG) tasks in seq2seq pre-trained language models (BART, T5), and proposes a probability-based decoding selection algorithm called DESEL to improve generation quality.
Rethinking Negative Pairs in Code Search
Haochen Li (Nanyang Technological University), Chunyan Miao (Nanyang Technological University)
RetrievalTransformerContrastive LearningText
🎯 What it does: Improve the handling of negative samples in code retrieval tasks by introducing the Soft-InfoNCE loss, which assigns different weights to negative samples to enhance the model's ability to distinguish them.
Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models
Xiaolei Wang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
Recommendation SystemExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Systematically evaluate ChatGPT's performance on conversational recommendation tasks, and propose an interactive evaluation framework called iEvaLM based on LLMs to simulate real users and conduct more realistic evaluations of recommendation systems.
Rethinking Word-Level Auto-Completion in Computer-Aided Translation
Xingyu Chen (Shanghai Jiao Tong University), Rui Wang (Tencent AI Lab)
TransformerText
🎯 What it does: Redefine the evaluation criteria for Word-Level Auto Completion (WLAC), introducing the 'consistency' criterion, and propose a joint training method based on machine translation, significantly improving WLAC performance.
Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System
Weizhou Shen (Sun Yat-sen University), Wei Bi (Tencent)
GenerationRetrievalTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes an end-to-end task-oriented dialogue system, MK-TOD, which jointly trains a retriever and a generator using maximum marginal likelihood, and introduces retrieval-related meta-knowledge to alleviate the retrieval-generation imbalance problem.