Annual Meeting of the Association for Computational Linguistics Β· 412 papers
Just Like a Human Would, Direct Access to Sarcasm Augmented with Potential Result and Reaction
Changrong Min, Hongfei Lin (Dalian University Of Technology)
CodeClassificationGraph Neural NetworkText
π― What it does: This study proposes a novel sarcasm detection method called SD-APRR, which augments sarcastic text with potential outcomes and human reactions inferred by COMET, to more comprehensively express the negative context within sarcasm.
KILM: Knowledge Injection into Encoder-Decoder Language Models
Yan Xu (Amazon Alexa AI), Dilek Hakkani-Tur
CodeGenerationRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
π― What it does: Utilizing continuous pre-training, inject structured knowledge from Wikipedia, including entity links and their short descriptions, into the Encoder-Decoder language model BART, enabling the model to acquire entity-related knowledge without modifying the architecture or increasing parameters.
π― What it does: Propose a k-Nearest Neighbor Transfer Learning (k NN-TL) framework to enable continuous utilization of the parent model's knowledge throughout the initialization, training, and inference processes in low-resource neural machine translation.
KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations
Myeongjun Jang (University of Oxford), Oana-Maria Camburu (University College London)
CodeExplainability and InterpretabilityAdversarial AttackLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
π― What it does: Proposed a fast consistency attack eKnowIA based on external knowledge, and introduced the KNOW method to alleviate NLE inconsistencies through knowledge injection;
Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Joel Jang (Korea Advanced Institute of Science and Technology), Minjoon Seo (Korea Advanced Institute of Science and Technology)
CodeSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Achieve knowledge forgetting by performing gradient ascent (i.e., maximizing loss) on the target token sequence of pre-trained language models, thereby eliminating extractable private information without retraining the model.
Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversations
Yang Deng, Wai Lam (The Chinese University of Hong Kong)
CodeTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Propose a hybrid proactive emotional support dialog system KEMI, integrating knowledge retrieval with multi-task generation to enhance system proactiveness and information quality.
Knowledgeable Parameter Efficient Tuning Network for Commonsense Question Answering
Ziwang Zhao (Beijing University of Posts and Telecommunications), Yequan Wang (Beijing Academy of Artificial Intelligence)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Propose a knowledge-aware parameter-efficient fine-tuning network called KPE, which injects external knowledge into a frozen pre-trained language model through parameter-sharing adapters to enhance commonsense question answering performance.
Language model acceptability judgements are not always robust to context
Koustuv Sinha (Meta AI), Adina Williams (Meta AI)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Investigating the robustness of large language models in judging acceptability under different contexts (length, structural similarity, presence of grammatical errors).
CodeExplainability and InterpretabilityRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Apply 'data intervention' using a small number (10 samples) of gender bias training data on pre-trained language models, followed by fine-tuning to reduce gender bias in the models.
CodeKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
π― What it does: Leverage large-scale language models to generate chain-of-thought examples, followed by fine-tuning small models to achieve significant capabilities in complex reasoning tasks;
π― What it does: Conduct large-scale correlation analysis between automatic consistency metrics and human judgments, sampling thousands of model-free generated topics on three corpora, and designing fine-grained user studies to examine the consistency between metrics and human evaluations.
Learning Better Masking for Better Language Model Pre-training
Dongjie Yang (Shanghai Jiao Tong University), Hai Zhao (Shanghai Jiao Tong University)
CodeOptimizationRepresentation LearningData-Centric LearningTransformerLarge Language ModelContrastive LearningText
π― What it does: This paper studies and proposes two time-varying masking strategies (Masking Ratio Decay and POS-Tagging Weighted Masking) to improve the pre-training process of MLM models such as BERT.
CodeRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Proposes the Skill-LM framework, achieving the injection of strict arithmetic reasoning into large language models (LLMs) without compromising language capabilities.
Learning to Generate Equitable Text in Dialogue from Biased Training Data
Anthony Sicilia (University of Pittsburgh), Malihe Alikhani (University of Pittsburgh)
CodeGenerationSupervised Fine-TuningText
π― What it does: Studied methods to achieve fairness (equity) in dialogue generation, providing a formal definition of fairness (score parity). It was proven through computational learning theory that minimizing test divergence can achieve fairness, enabling fair dialogue generation without altering biased training data.
Learning to Imagine: Visually-Augmented Natural Language Generation
Tianyi Tang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelDiffusion modelTextMultimodality
π― What it does: Proposed the LIVE method, which achieves visual-enhanced natural language generation by dynamically generating and filtering visual images for input sentences, and then fusing them with pre-trained language models.
LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development
Ilias Chalkidis, Anders SΓΈgaard (University Of Copenhagen)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Construct multilingual English legal corpus LeXFiles and release the LegalLAMA probing benchmark, training and evaluating two LexLM legal PLMs as well as other models' pre-training, probing, and downstream performance.
Linear Classifier: An Often-Forgotten Baseline for Text Classification
Yu-Chen Lin (National Taiwan University), Chih-Jen Lin (National Taiwan University)
CodeClassificationTransformerLarge Language ModelText
π― What it does: Compare the performance of linear SVM and BERT in text classification, highlighting the importance of linear classifiers as essential baselines.
π― What it does: The study uses automatically generated dependency parsing and abstract meaning representation (AMR) graphs as additional features to enhance the model in cross-domain few-shot relation extraction tasks.
LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically Constructed from Live Streaming
Jingsheng Gao (Shanghai Jiao Tong University), Baoyuan Wang (Xiaobing.AI)
CodeGenerationRetrievalTransformerLarge Language ModelVideoTextBenchmark
π― What it does: Constructed a Chinese personalized dialogue dataset based on live video called LiveChat, and designed and evaluated retrieval-based and generative dialogue models on it.
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Proposed and constructed the MAD-TSC datasetβthe first multilingual aligned news target sentiment analysis (TSC) dataset, and evaluated various TSC methods based on pre-trained language models on it;
π― What it does: In resource-scarce dialects/minority languages (Gronings, Frisian, Bezemer, and Nasari), transfer learning via a self-supervised pre-trained model (XLS-R) is applied, combined with data augmentation using self-training and TTS-generated synthetic speech to improve automatic speech recognition (ASR) word error rate (WER).
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
Jack FitzGerald (Amazon), Prem Natarajan (Capital One)
CodeClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Constructed a multilingual natural language understanding (NLU) dataset named MASSIVE with 1 million examples, covering 51 languages, 18 domains, 60 intents, and 55 slots;
π― What it does: This paper proposes a multilingual lexical hierarchy specialization method, which uses the multilingual synonym relations from BabelNet to perform one-time alignment and fine-tuning on multilingual Transformers (mBERT, XLM-R), thereby generating superior static word vectors.
mCLIP: Multilingual CLIP via Cross-lingual Transfer
Guanhua Chen (Southern University of Science and Technology), Wenping Wang (Texas Aamp;M University)
CodeRetrievalKnowledge DistillationTransformerVision Language ModelContrastive LearningImageText
π― What it does: This paper proposes mCLIP, a dual-stream multilingual vision-language pre-training model that aligns CLIP with a multilingual text encoder through triangular cross-modal knowledge distillation.
MDACE: MIMIC Documents Annotated with Code Evidence
Hua Cheng (3M Health Information Systems), Matthew Gormley
CodeClassificationExplainability and InterpretabilityConvolutional Neural NetworkBiomedical DataElectronic Health RecordsBenchmark
π― What it does: Constructed and made public the MDACE dataset, which contains text evidence spans from diagnosis and surgical codes in MIMIC-III clinical records;
Measuring Consistency in Text-based Financial Forecasting Models
Linyi Yang (Westlake Institute for Advanced Study), Yue Zhang (Westlake University)
CodeTransformerSupervised Fine-TuningTextTabularFinance Related
π― What it does: In financial text prediction tasks, the FinTrust evaluation tool is proposed to detect model logical consistency under input transformations that preserve meaning, combined with two evaluation settings: β Consistency evaluation of masked word prediction in pre-trained language models (PLMs); β‘ Evaluation of prediction behavior after applying four types of logical consistency transformations (negation, symmetry, additivity, transitivity) to text-based financial prediction models.
Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations
Chenglei Si (University of Maryland), He He (New York University)
CodeClassificationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Investigated the preferences of large language models for different features during in-context learning (ICL), by constructing 'underdetermined' demonstration data to evaluate model feature bias, and attempted various intervention methods (instructions, label words, explanations, etc.) to guide models toward specific features.
CodeVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
π― What it does: Evaluated and compared the performance of multiple Vision-Language Models (VLMs) on fine-grained vision-and-language understanding benchmarks, highlighting the role of explicit object modeling and specific loss functions in enhancing fine-grained capabilities.
Measuring the Effect of Influential Messages on Varying Personas
Chenkai Sun (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
CodeGenerationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: The study proposes a personalized response prediction task for news media, predicting the emotional polarity, intensity, and textual responses that different individuals may generate after viewing news.
π― What it does: Propose a meta-learning based domain adaptive few-shot fake news detection framework, MetaAdapt, which achieves cross-domain adaptation using source domain tasks and a very small number of target domain samples.
MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models
Masoud Monajatipoor (UCLA), Kai-Wei Chang (UCLA)
CodeComputational EfficiencyKnowledge DistillationRepresentation LearningMeta LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelContrastive LearningMultimodality
π― What it does: This paper proposes the MetaVL model, which transfers the few-shot context learning ability trained in language models to vision-language tasks, achieving strong few-shot performance with a smaller model.
Wei Liu (Huazhong University of Science and Technology), Yang Qiu (Huazhong University of Science and Technology)
CodeExplainability and InterpretabilityRecurrent Neural NetworkMixture of ExpertsText
π― What it does: Propose the Multi-Generator (MGR) framework, which jointly trains multiple sets of generators with a single predictor to achieve self-explanatory NLP models;
π― What it does: Propose a new sentence embedding method called mi CSE, which leverages mutual information in attention layers to enforce structural consistency between different dropout views, and combines Momentum Contrastive Learning for self-supervised training.
MIL-Decoding: Detoxifying Language Models at Token-Level via Multiple Instance Learning
Xu Zhang (Wangxuan Institute of Computer Technology, Peking University), Xiaojun Wan (Wangxuan Institute of Computer Technology, Peking University)
CodeClassificationSafty and PrivacyRecurrent Neural NetworkLarge Language ModelText
π― What it does: This study proposes the MIL-Decoding method, which assigns toxicity scores to each candidate word during language model decoding through a multi-instance learning (MIL) network, achieving token-level detoxification by combining the original language model probabilities.
Minding Language Modelsβ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker
Melanie Sclar (University of Washington), Yulia Tsvetkov
CodeExplainability and InterpretabilityRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTextGraphRetrieval-Augmented Generation
π― What it does: Propose a training-free, symbolic method called SYMBOLICTOM that explicitly tracks multi-agent belief states using graph structures and leverages existing large language models for reading comprehension reasoning.
MIReAD: Simple Method for Learning High-quality Representations from Scientific Documents
Anastasiia Razdaibiedina (University of Toronto), Aleksandr Brechalov (University of Toronto)
CodeClassificationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
π― What it does: Fine-tune SciBERT using a journal classification task, learning high-quality scientific document representations based solely on paper titles and abstracts.
π― What it does: Through a case study on the E2E ST pre-training and fine-tuning framework, this paper investigates the impact of modal adaptation and regularization on performance, and proposes an auxiliary branch method with adjustable modal adaptation (TAB).
Timon Ziegenbein (Leibniz University Hannover), Henning Wachsmuth (Leibniz University Hannover)
CodeClassificationTransformerLarge Language ModelText
π― What it does: This study systematically models appropriate language in online arguments, proposing a 14-dimensional hierarchical classification and manually annotating 2191 arguments;
π― What it does: Construct sentence graphs and subgraph sets, build a document-subgraph heterogeneous graph using shared subgraphs, then encode the graph with GCN for text coherence evaluation and automatic writing scoring.
Modular Visual Question Answering via Code Generation
Sanjay Subramanian (University of California Berkeley), Dan Klein (University of California Berkeley)
CodeExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodality
π― What it does: Generating Python programs using large language models (e.g., code-davinci-002) to decompose visual tasks into calling pre-trained visual modules (query, get_pos, find_matching_image) and combining them through code logic to accomplish visual question answering;
MolXPT: Wrapping Molecules with Text for Generative Pre-training
Zequn Liu (Peking University), Tie-Yan Liu (Microsoft Research AI4Science)
CodeDrug DiscoveryTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBiomedical Data
π― What it does: Designed and pre-trained MolXPT, a unified generative pre-trained Transformer capable of processing scientific text, SMILES, and their wrapped sequences simultaneously.
MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions
Hao Sun (Tsinghua University), Minlie Huang (Xiaomi AI Lab)
CodeGenerationExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: Propose the MORALDIAL framework, which constructs moral discussion data to train and evaluate the morality of open-domain dialogue systems; build four subtasks (moral answer, explanation, revision, reasoning), and adopt multi-task learning to train on DialoGPT/Blenderbot; design a no-reference evaluation method based on answer-RoT consistency scoring.
Zihao Yue (Renmin University of China), Qin Jin (Renmin University of China)
CodeTransformerVision Language ModelContrastive LearningVideoTextMultimodalityBenchmarkRetrieval-Augmented Generation
π― What it does: Constructed a large-scale Chinese movie understanding benchmark named Movie101, and proposed two tasks: Movie Clip Narration (MCN) and Temporal Narration Localization (TNG), along with a new evaluation metric, MNScore, which shows high correlation with human assessments.
mPMR: A Multilingual Pre-trained Machine Reader at Scale
Weiwen Xu (Chinese University of Hong Kong), Lidong Bing (DAMO Academy, Alibaba Group)
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Developed a multilingual machine reading comprehension pre-training framework called mPMR, which inherits and enhances the natural language understanding capabilities of existing multilingual pre-training models (e.g., XLM-R) by generating MRC-style training data on Wikipedia links.
MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction
Wang Jing, Xiaoli Li (Nanyang Technological University)
CodeRetrievalTransformerVision Language ModelVideoText
π― What it does: Proposed an MS-DETR based on the proposal-based DETR framework for natural language video localization, combining a multi-scale vision-language encoder with an anchor-guided temporal decoder to achieve efficient temporal interaction.
π― What it does: Propose Multi-grAined KnowlEdge Retriever (MAKER), which decouples knowledge retrieval from generation, achieving multi-grained retrieval through entity and attribute selectors, and trains the retriever via knowledge distillation from the generator.
Mengze Li (Zhejiang University), Fei Wu (Hikvision Research Institute)
CodeRecognitionExplainability and InterpretabilityConvolutional Neural NetworkGraph Neural NetworkTransformerVision-Language-Action ModelVideoTextMultimodalityChain-of-Thought
π― What it does: Studied the multimodal action chain abductive reasoning task (MAR), and proposed a complete framework for target event localization and subsequent action chain inference.
Liqiang Jing (Shandong University), Liqiang Nie (Shandong University)
CodeGenerationGraph Neural NetworkTransformerLarge Language ModelVision Language ModelMultimodalityRetrieval-Augmented Generation
π― What it does: Proposed the TEAM model, aiming to automatically generate explanatory sentences for multimodal sarcastic social posts (image + caption);
Multimodal Persona Based Generation of Comic Dialogs
Harsh Agrawal (IIT Delhi), Mausam (IIT Delhi)
CodeGenerationTransformerLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: This paper proposes a manga dialogue generation framework based on multimodal (text + image) and character features, and constructs the COMSET dataset containing 13 comics, 54K comic panels, and 200+ characters with their personality descriptions.
Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis
Xuming Hu (Tsinghua University), Philip S. Yu (University of Illinois at Chicago)
CodeData SynthesisRetrievalRepresentation LearningConvolutional Neural NetworkTransformerVision Language ModelMultimodalityRetrieval-Augmented Generation
π― What it does: Propose a cross-modal retrieval and synthesis framework that retrieves textual and visual evidence for objects, sentences, and entire graphs, and performs relation extraction by fusing multi-modal information through multi-head attention.
MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering
Vaishali Pal (University of Amsterdam), Maarten de Rijke (University of Amsterdam)
CodeTransformerLarge Language ModelSupervised Fine-TuningTabular
π― What it does: This paper proposes a multi-table question-answering model called MultiTabQA, which can directly generate structured table answers from natural language questions or SQL queries and multiple tables;
Multitask Pretraining with Structured Knowledge for Text-to-SQL Generation
Robert Giaquinto (AWS AI Labs), Xiaofei Ma (AWS AI Labs)
CodeGenerationRepresentation LearningTransformerLarge Language ModelTextTabular
π― What it does: Propose a multi-task pre-training framework called STAMP, specifically designed for the text-to-SQL generation task, incorporating joint learning of tables, SQL code, and natural language within an encoder-decoder structure.
Yongqi Li (Hong Kong Polytechnic University), Wenjie Li (Hong Kong Polytechnic University)
CodeRetrievalTransformerLarge Language ModelText
π― What it does: Propose the MINDER framework, which leverages autoregressive language models to generate multi-view identifiers (title, substring, pseudo queries) and achieve generative retrieval;
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Proposed NEUROSTRUCTURAL DECODING, an algorithm that integrates syntactic constraints into beam search decoding, and implemented an adaptive parser to handle incomplete sentences.
Iftitahu Nimah, Mykola Pechenizkiy (Eindhoven University of Technology)
CodeGenerationTextBenchmark
π― What it does: Designed and evaluated a multi-level assessment framework called Metric Preference Checklist to measure the effectiveness of automatic evaluation metrics in three types of natural language generation tasks (summarization, dialogue, and controlled generation).
NollySenti: Leveraging Transfer Learning and Machine Translation for Nigerian Movie Sentiment Classification
Iyanuoluwa Shode (Montclair State University), Anna Feldman (Montclair State University)
CodeClassificationDomain AdaptationTransformerLarge Language ModelText
π― What it does: Constructed the NollySenti movie review sentiment dataset, conducting cross-domain and cross-lingual sentiment classification experiments using transfer learning and machine translation.
π― What it does: By aligning wikiHow linear scripts with video multimodal information, automatically generate non-sequential graph scripts that capture relationships between optional and interchangeable steps.
Nonlinear Structural Equation Model Guided Gaussian Mixture Hierarchical Topic Modeling
HeGang Chen, Yanghui Rao (Sun Yat-sen University)
CodeRepresentation LearningAuto EncoderText
π― What it does: Propose a deep hierarchical topic model called NSEM-GMHTM, which explicitly models the hierarchical and symmetric dependencies of topics through a nonlinear structural equation model (NSEM) and Gaussian Mixture Prior (GMM), thereby enhancing topic coherence and the rationality of hierarchical structures.
π― What it does: Proposes a transferable backdoor attack called NOTABLE, achieving high success rates in attacking prompt-based NLP models across arbitrary tasks and prompt strategies.
π― What it does: Proposed an unknown-aware training method that enhances the detection capability of unknown relations in open-set relation extraction by dynamically synthesizing more challenging negative samples.
π― What it does: This paper proposes the OpenSR training system, which utilizes phoneme space alignment pre-trained from unlabeled multi-modal audio-visual data in high-resource domains. It trains a decoder directly applicable to visual or audio-visual recognition using only target domain annotated audio data, achieving zero-shot speech recognition.
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional Experts
Yunshui Li (Shenzhen Institute of Advanced Technology Chinese Academy of Sciences), Yongbin Li (DAMO Academy Alibaba Group)
CodeGenerationRetrievalRepresentation LearningTransformerMixture of ExpertsVision Language ModelMultimodality
π― What it does: Propose the PaCE framework, decomposing multimodal dialogue into five experts: CAPTION, CONTEXT, IMAGE, GROUNDING, and GENERATION, achieving unified multimodal dialogue pre-training through progressive pre-training.
PAD-Net: An Efficient Framework for Dynamic Networks
Shwai He (University of Maryland), Dacheng Tao (University of Sydney)
CodeComputational EfficiencyConvolutional Neural NetworkTransformerMixture of ExpertsImageText
π― What it does: Propose a Partial Dynamic Network (PAD-Net) framework that converts redundant dynamic parameters in dynamic networks into static parameters to reduce deployment costs and improve performance.
π― What it does: This paper studies Person Attribute Extraction in Dialogues (PAED), constructs a high-quality PersonaExt dataset, and proposes a generative zero-shot learning framework based on Meta-VAE hard negative sampling and contrastive structured constraints.
PAL to Lend a Helping Hand: Towards Building an Emotion Adaptive Polite and Empathetic Counseling Conversational Agent
Kshitij Mishra (Indian Institute of Technology Patna), Asif Ekbal (Indian Institute of Technology Patna)
CodeGenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: Developed an emotionally adaptive, polite, and empathetic counseling dialogue system called PAL, aiming to provide a more engaging and friendly interactive experience for online psychological counseling.
ParaLS: Lexical Substitution via Pretrained Paraphraser
Jipeng Qiang (Yangzhou University), Yi Zhu (Yangzhou University)
CodeGenerationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Generate synonym replacement candidates that preserve word meaning using a pre-trained paraphraser and filter them through specific decoding strategies.
π― What it does: This paper proposes two parameter-efficient fine-tuning (PEFT) methods: PaFi (which utilizes the amplitude of pre-trained parameters to select sparse masks) and HiWi (which directly applies adapters to pre-trained weights/bias).
Peeking inside the black box: A Commonsense-aware Generative Framework for Explainable Complaint Detection
Apoorva Singh (IIT Patna), Sriparna Saha (IIT Patna)
CodeGenerationExplainability and InterpretabilityTransformerLarge Language ModelReinforcement LearningText
π― What it does: Proposed an explainable complaint detection framework and the X-CI dataset, converting multi-task prediction (complaint, severity, emotion, polarity, causal reason) into a text-to-text generation problem.
π― What it does: Explored the potential correlation between peer labels in hierarchical text classification and leveraged peer label collaboration to enhance classification performance.
Personality Understanding of Fictional Characters during Book Reading
Mo Yu (WeChat AI), Jie Zhou (WeChat AI)
CodeClassificationTransformerLarge Language ModelText
π― What it does: Constructed the PERSONET dataset, utilizing online reading notes instead of complete book texts, to study the fine-grained personality prediction task based on historical context during reading.
PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English
Jianfeng Chi (Meta AI), Kai-Wei Chang (University of California Los Angeles)
CodeDomain AdaptationSafty and PrivacyTransformerSupervised Fine-TuningTextBenchmark
π― What it does: Constructed PLUE, a multi-task evaluation benchmark covering six privacy policy language understanding tasks including text classification, question answering, semantic parsing, and named entity recognition, and collected a 332M-word privacy policy corpus for domain-specific pre-training.
π― What it does: This paper proposes a cross-prompt automatic essay scoring (AES) method called PMAES, which enhances scoring performance by using prompt mapping contrastive learning to make representations of source prompts and target prompts more consistent.
Pre-training Multi-party Dialogue Models with Latent Discourse Inference
Yiyang Li (Shanghai Jiao Tong University), Hai Zhao (Shanghai Jiao Tong University)
CodeGenerationRepresentation LearningTransformerLarge Language ModelContrastive LearningText
π― What it does: Pre-train a dialogue model with better discourse structure awareness by using two-stage unsupervised reasoning (EM and VI) to infer implicit argumentation structures (response relationships) in multi-party dialogues.
Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference
Junhao Zheng (South China University of Technology), Haibin Chen (South China University of Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose a fine-tuning method based on causal inference called Causal Effect Tuning (CET), which recovers pre-trained knowledge through KNN-weighted joint prediction, avoiding catastrophic forgetting and negative transfer.
Pretrained Bidirectional Distillation for Machine Translation
Yimeng Zhuang (Samsung Research China - Beijing), Mei Tu (Samsung Research China - Beijing)
CodeGenerationKnowledge DistillationTransformerLarge Language ModelText
π― What it does: Propose the pre-trained bidirectional distillation (PBD) method, which distills the bidirectional language knowledge of masked language models into NMT encoders and decoders in one go during training;
Privacy-Preserving Domain Adaptation of Semantic Parsers
Fatemehsadat Mireshghallah (University of California, San Diego), Richard Shin (Microsoft Semantic Machines)
CodeDomain AdaptationSafty and PrivacyTransformerLarge Language ModelText
π― What it does: In task-oriented dialogue systems, synthetic user dialogue semantic parsing data is generated using differential privacy (DP) techniques, which is used to enhance the performance of low-resource semantic parsers while ensuring user privacy and diagnosing system defects.
Product Question Answering in E-Commerce: A Survey
Yang Deng, Wai Lam (JD.com)
CodeRetrievalGraph Neural NetworkTransformerMixture of ExpertsTextMultimodalityReview/Survey PaperRetrieval-Augmented Generation
π― What it does: A systematic review of product question answering (PQA) in the e-commerce domain, covering four mainstream question settings (opinion-based, extractive, retrieval-based, and generative), summarizing related methods, datasets, evaluation metrics, and identifying key challenges and future research directions.
π― What it does: This paper proposes a technique that can compress pre-trained language models (PLM) to high sparsity rates through a single static pruning (SMP) without requiring fine-tuning.
PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism
Yongkang Liu (Northeastern University), Hinrich SchΓΌtze (Northeastern University)
CodeGenerationRecurrent Neural NetworkText
π― What it does: Proposed a PVGRU and PVHD structure based on a pseudo variational mechanism to enhance the diversity and relevance of multi-turn dialogue generation.
CodeGenerationData SynthesisAI Code AssistantGraph Neural NetworkTransformerLarge Language ModelText
π― What it does: Propose a method to enhance Python code generation by asking clarifying questions, construct the CodeClarQA dataset, and implement an interactive generation pipeline
Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling
Mingzhu Cai (Baidu Inc.), Hua Wu (Baidu Inc.)
CodeGenerationRetrievalData-Centric LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Proposes an unsupervised query enhancement method called QKConv, which automatically generates queries and generates responses based on external knowledge through joint training in knowledge-intensive dialogues.
Query-Efficient Black-Box Red Teaming via Bayesian Optimization
Deokjae Lee (Seoul National University), Hyun Oh Song (Seoul National University)
CodeOptimizationSafty and PrivacyAdversarial AttackHyperparameter SearchTransformerImageText
π― What it does: For red team testing against black-box generative models, this paper proposes BRT, a query-efficient method based on Bayesian optimization, to efficiently discover diverse adversarial examples within a limited query budget.
π― What it does: Constructed the first Tigrinya (Tigrinya) question-answering dataset TiQuAD, and conducted various experiments (monolingual, cross-lingual, cross-language, and multilingual), demonstrating the feasibility of question-answering tasks on low-resource languages.
π― What it does: Propose a two-stage RSMI framework, first obtaining robust hidden representations through randomized smoothing, then suppressing adversarial perturbations via gradient-guided mask inference to enhance adversarial robustness in text classification.
π― What it does: Proposed a fine-grained semantic matching method for zero-shot relation extraction, decomposing sentence matching into entity matching and context matching, and reducing interference from redundant information through context feature distillation.
Shuofei Qiao (Zhejiang University), Huajun Chen (National University Of Singapore)
CodeTransformerLarge Language ModelTextReview/Survey PaperBenchmarkChain-of-Thought
π― What it does: Reviews reasoning methods under language model prompts, categorizes them into strategy enhancement and knowledge enhancement, and provides method classification, comparison, and resource organization.
ReCode: Robustness Evaluation of Code Generation Models
Shiqi Wang (AWS AI Labs), Bing Xiang (AWS AI Labs)
CodeGenerationTransformerLarge Language ModelTextBenchmark
π― What it does: Propose the ReCode benchmark, defining over 30 natural document, function name, syntax, and format perturbations, and evaluate the robustness of code generation models using worst-case metrics.
π― What it does: This paper proposes two large-scale multilingual relation extraction datasetsβSRED FM (gold-standard, 18 languages, 400 relations, 44M triplets) and RED FM (human-verified, 7 languages, 32 relations, ~1M triplets)βand trains the first multilingual end-to-end relation extraction model, mREBEL, based on these datasets;
CodeGenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelMultimodalityBenchmark
π― What it does: This paper investigates the ambiguity problem in text-to-image generation models, constructing a TAB benchmark dataset containing various types of ambiguity, and proposes a pluggable TIED framework. TIED generates clarifying questions or visual scene descriptions via language models, leverages user interaction to resolve ambiguity, inputs the disambiguated prompts into text-to-image models, and evaluates improvements in image realism and fairness.
π― What it does: By designing controlled experiments, 36 language learners of varying proficiency levels were recruited to annotate data across four NLP tasks in English, Korean, and Indonesian (sentiment analysis, natural language inference, named entity recognition, and machine reading comprehension), while assessing annotation quality and improvements in learners' language proficiency.
CodeData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: This paper decomposes Chinese Spelling Correction (CSC) into a language model and an error model, discovering that traditional BERT fine-tuning overfits the error model and underfits the language model, leading to poor generalization on unseen error patterns.