Annual Meeting of the Association for Computational Linguistics Β· 412 papers
Rethinking Multimodal Entity and Relation Extraction from a Translation Point of View
Changmeng Zheng (Hong Kong Polytechnic University), Qing Li (Hong Kong Polytechnic University)
CodeRecognitionImage TranslationTransformerVision Language ModelDiffusion modelContrastive LearningMultimodality
π― What it does: This paper re-examines the task of multi-modal entity and relation extraction, proposing to address alignment errors between text and images by treating text-image pairs as mutual translations. It achieves multi-modal reverse translation through generative diffusion models and constructs a high-resource bridging multi-modal bias estimator to obtain fine-grained alignment confidence.
Retrieval-free Knowledge Injection through Multi-Document Traversal for Dialogue Models
Rui Wang (Harbin Institute of Technology), Ruifeng Xu (Huawei Noah's Ark Lab)
CodeGenerationGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose a retrieval-agnostic knowledge injection method KiDG, which automatically converts knowledge documents into simulated dialogues through multi-document traversal to enhance the knowledge generation capability of dialogue models.
π― What it does: Propose a dual mask autoencoder (DupMAE) framework that jointly trains the embeddings of CLS and regular tokens to generate more optimal retrieval semantic representations.
Revisiting Commonsense Reasoning in Machine Translation: Training, Evaluation and Challenge
Xuebo Liu (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeGenerationTransformerTextBenchmark
π― What it does: This paper systematically studies the common-sense reasoning (CR) capability in neural machine translation (NMT), exploring pre-training, evaluation methods, and challenges in enhancing CR.
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
Yixin Liu (Yale University), Dragomir Radev (Yale University)
CodeGenerationTextBenchmark
π― What it does: Propose the ACU (Atomic Content Unit) fine-grained evaluation protocol and construct the RoSE large-scale human evaluation benchmark based on this protocol, conducting systematic comparisons of multiple evaluation protocols and automatic evaluation metrics.
Revisiting Token Dropping Strategy in Efficient BERT Pretraining
Qihuang Zhong (Wuhan University), Dacheng Tao (University of Sydney)
CodeComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelText
π― What it does: Studied the semantic loss problem caused by the token dropping strategy and proposed a semantic consistency learning (SCTD) method to improve pre-training efficiency.
RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs
Afra Feyza Akyurek, Niket Tandon (Allen Institute for Artificial Intelligence)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: Propose the RL4F framework, enabling a language model to generate natural language feedback to improve the output of another fixed task model (e.g., GPT-3).
Robust Multi-bit Natural Language Watermarking through Invariant Features
KiYoon Yoo (Seoul National University), Nojun Kwak (Seoul National University)
CodeLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose a multi-bit watermark embedding and extraction framework based on text semantics and syntactic invariant features for copyright protection and leakage tracking of natural language content.
π― What it does: Proposed a robust short text clustering model RSTC, achieving adaptive clustering for imbalanced and noisy data through two modules: pseudo-label generation and robust representation learning.
S2ynRE: Two-stage Self-training with Synthetic data for Low-resource Relation Extraction
Benfeng Xu (University of Science and Technology of China), Zhendong Mao (Institute of Artificial Intelligence, Hefei Comprehensive National Science Center)
CodeClassificationData SynthesisKnowledge DistillationTransformerLarge Language ModelText
π― What it does: Generate synthetic training samples that match the target domain using large language models, and enhance low-resource relation extraction performance by alternately learning from synthetic data and golden data through a two-stage self-training algorithm.
SafeConv: Explaining and Correcting Conversational Unsafe Behavior
Mian Zhang (Soochow University), Dong Yu (Tencent AI Lab)
CodeSafty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
π― What it does: Constructed a Chinese dialogue safety dataset named SAFECONV containing dialogue-level safety labels, unsafe spans, and safe alternative responses, and trained safety detectors, unsafe span annotators, and context rewriters on it.
SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models
Akshita Jha (Virginia Tech), Sunipa Dev (Google Research)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Leverage large language models (PaLM, GPT-3, T0) to generate stereotypes at global and regional levels, which are subsequently validated for authenticity and offensiveness through a globally diverse human review, constructing SeeGULL, a large-scale, cross-cultural stereotype benchmark dataset.
Seen to Unseen: Exploring Compositional Generalization of Multi-Attribute Controllable Dialogue Generation
Weihao Zeng (Beijing University of Posts and Telecommunications), Weiran Xu (Meituan)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes a decoupled controllable dialogue generation model based on prompts, DCG, which can learn from seen attribute values and achieve compositional generalization on unseen attribute combinations.
Self-Edit: Fault-Aware Code Editor for Code Generation
Kechi Zhang (Peking University), Zhi Jin (Peking University)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: This paper proposes a generation-editing (Self-Edit) framework that leverages large language models (LLMs) to first generate competition problem code, then obtain error information through execution examples, generates supplementary comments, and subsequently modifies the generated code using a specifically trained fault-aware code editor, ultimately producing higher quality programs.
Zhilei Hu (University of Chinese Academy of Sciences), Xueqi Cheng (Institute of Computing Technology, Chinese Academy of Sciences)
CodeClassificationRecurrent Neural NetworkGraph Neural NetworkLarge Language ModelText
π― What it does: Propose the SemSIn model, which utilizes semantic graphs derived from AMR parsing, combining event-centric structures and event-related structures to enhance event causality recognition.
π― What it does: This paper proposes an unsupervised contextual common sense knowledge graph (CCKG) construction method to supplement implicit common sense reasoning in argumentative texts, thereby improving performance on knowledge-intensive argument tasks.
SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation
Junkai Zhou (Institute of Computing Technology Chinese Academy of Sciences), Xueqi Cheng (Institute of Computing Technology Chinese Academy of Sciences)
CodeGenerationComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
π― What it does: Proposes a two-phase method (SimOAP) that enhances the coherence and consistency of persona-based dialogue generation through large-scale oversampling and post-evaluation.
π― What it does: This paper proposes a simple and effective unsupervised speech translation framework. It first generates pseudo labels using unsupervised ASR, MT, and TTS models, then trains end-to-end S2TT and S2ST models. The recognition performance is further improved through unsupervised domain adaptation of wav2vec 2.0.
Simple Augmentations of Logical Rules for Neuro-Symbolic Knowledge Graph Completion
Ananjan Nandi (Indian Institute of Technology Delhi), Mausam (Indian Institute of Technology Delhi)
CodeExplainability and InterpretabilityRepresentation LearningRecurrent Neural NetworkGraph Neural NetworkGraph
π― What it does: Investigate simple and general rule expansion techniques to improve the rule coverage and performance of neural symbolic knowledge graph completion (NS-KGC) models. The method converts existing rules into abductive forms, generates equivalent rules using inverse relationships, and creates new rules through local random walks. Subsequently, PCA scoring is used to filter low-quality rules, ultimately obtaining a high-quality, high-coverage rule set.
π― What it does: Analyze the preference of Transformer and LSTM during random initialization and training for low-sensitivity Boolean functions, and evaluate their generalization ability on sparse Boolean functions.
π― What it does: Propose the SLUE Phase-2 benchmark suite, which includes four diversified spoken language understanding tasks (dialogue act classification, question answering, summarization, and named entity localization), and provides datasets, human annotations, pipeline and end-to-end baseline models, as well as evaluation metrics for each task.
π― What it does: Propose an over-parameterization (OPF) method for small pre-trained language models during the fine-tuning phase through matrix decomposition, significantly enhancing model performance without increasing inference latency.
Social-Group-Agnostic Bias Mitigation via the Stereotype Content Model
Ali Omrani (University of Southern California), Morteza Dehghani (University of Southern California)
CodeRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Propose a community-irrelevant bias mitigation method based on the social psychology stereotype content model (SCM), implemented on both static word embeddings and large language models;
Soft Alignment Objectives for Robust Adaptation of Language Generation
Michal Ε tefΓ‘nik (Masaryk University), Petr Sojka (Masaryk University)
CodeGenerationDomain AdaptationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose a soft alignment objective based on domain-agnostic semantic similarity to improve domain adaptation in language generation models and reduce catastrophic forgetting.
Soft Language Clustering for Multilingual Model Pre-training
Jiali Zeng (Tencent Inc), Jie Zhou (Tencent Inc)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
π― What it does: Propose the XLM-P model, which enhances the model in multilingual pre-training by using context-retrieved soft prompts as dynamic language clustering information.
π― What it does: Explored how to use prompt-driven neural machine translation models to generate singable lyrics that comply with musical constraints.
SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres
Shumin Deng (National University of Singapore), Bryan Hooi (National University of Singapore)
CodeClassificationRecognitionTransformerLarge Language ModelText
π― What it does: Propose a model called SPEECH that combines energy networks and event-centric hyperspheres for event detection, classification, and relation extraction.
Stop Pre-Training: Adapt Visual-Language Models to Unseen Languages
Yasmine Karoui (Technical University of Munich), Karl Aberer (EPFL)
CodeRetrievalDomain AdaptationTransformerVision Language ModelContrastive LearningMultimodality
π― What it does: This paper proposes the CLiCoTEA method, which utilizes a small amount of parallel corpus obtained from machine translation. By aligning context word embeddings, it transfers existing monolingual vision-language pre-training models to unseen languages, achieving zero-shot cross-lingual visual tasks.
StoryWars: A Dataset and Instruction Tuning Baselines for Collaborative Story Understanding and Generation
Yulun Du (Columbia University), Lydia Chilton (Columbia University)
CodeRecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: This paper proposes a collaborative story dataset and multi-task benchmark named STORYWARS, which constructs 101 tasks covering understanding and generation. Based on this, we design and evaluate a two-stage instruction-tuned model called INSTRUCTSTORY.
Substitution-based Semantic Change Detection using Contextual Embeddings
Dallas Card (University of Michigan)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Use a contextual masked language model to obtain the top-k most probable substitutes for each word, statistically analyze their distribution, and detect semantic changes through Jensen-Shannon Divergence.
SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages
Philippe Laban (Salesforce AI), Chien-Sheng Wu
CodeGenerationTransformerLarge Language ModelTextBenchmark
π― What it does: Constructed the SWIPE dataset by leveraging the complete revision history of English Wikipedia and its simplified version to achieve document-level simplification alignment, and annotated over 5,000 document pairs with 19 fine-grained edit labels; simultaneously developed an automated editing recognition model;
Table and Image Generation for Investigating Knowledge of Entities in Pre-trained Vision and Language Models
Hidetaka Kamigaito (Nara Institute of Science and Technology), Taro Watanabe (Nara Institute of Science and Technology)
CodeGenerationTransformerVision Language ModelImageMultimodalityTabular
π― What it does: This paper proposes a table and image generation task to examine the retention of entity knowledge by visual-language models, and constructs the WikiTIG dataset.
CodeRecognitionData SynthesisTransformerVision Language ModelImageTextMultimodality
π― What it does: Proposed TableVLM, a multi-modal pre-training model and the ComplexTable dataset, for identifying table structures from table images.
CodeTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: Proposed the VGAMT (Visually Guided and Adapted Machine Translation) model, which combines lightweight adapters and visual projections with vision-conditioned masked language modeling (VMLM) and multimodal translation (MMT) dual objectives for joint training on a frozen strong text-based MT model (mBART), while constructing the CoMMuTE benchmark dataset to evaluate the role of images in disambiguation.
Tackling Modality Heterogeneity with Multi-View Calibration Network for Multimodal Sentiment Detection
Yiwei Wei (Tianjin University), Meng Chen (JD AI Research)
CodeClassificationTransformerVision Language ModelMultimodality
π― What it does: This paper proposes a Multi-View Calibration Network (MVCN), addressing the modality heterogeneity issue in text-image sentiment detection by designing three modules: text-guided fusion, emotional consistency constraints, and adaptive loss calibration.
TAGPRIME: A Unified Framework for Relational Structure Extraction
I-Hung Hsu (University of Southern California), Nanyun Peng (University of California, Los Angeles)
CodeTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Propose the TAGPRIME unified framework, which injects conditional and relational information into the input via priming to achieve relation structure extraction.
Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation
Kexin Yang (Alibaba Group), Jun Xie (Alibaba Group)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Propose a control text generation method called Tailor based on soft prompts, which achieves single-attribute and multi-attribute generation by learning continuous prompts for each attribute and enabling their combination.
CodeClassificationRecognitionRecurrent Neural NetworkTransformerLarge Language ModelText
π― What it does: This paper creates a new dataset called TBO (Target-Based Offensive Language Identification), which annotates English tweets with triplets of target, offensive phrases, and harmfulness, and conducts experimental evaluations on multiple models.
π― What it does: Designed a new temporal knowledge graph embedding model called TeAST, which maps relationships to an Archimedean spiral timeline to avoid mixing temporal information with entities, thereby better capturing patterns of how relationships evolve over time; and converts the quadruple completion problem into a third-order tensor completion task.
TemplateGEC: Improving Grammatical Error Correction with Detection Template
Yinghao Li (Beijing Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: This paper proposes a Chinese and English grammar error correction method called TemplateGEC, which combines Seq2Edit for error detection and Seq2Seq for error correction, and enhances robustness through consistency learning.
Daimeng Wei (Huawei Translation Service Center), Hao Yang (Huawei Translation Service Center)
CodeGenerationDomain AdaptationTransformerTextBiomedical Data
π― What it does: Propose Text Style Transfer for Back-Translation (TST BT), which improves translation quality for natural inputs by performing style transfer on source-side text from back-translation, making it closer to natural language corpora.
Text Style Transfer with Contrastive Transfer Pattern Mining
Jingxuan Han (University of Science and Technology of China), Zhendong Mao (University of Science and Technology of China)
CodeGenerationTransformerContrastive LearningText
π― What it does: This paper proposes a new method for text style transfer tasksβContrastive Transfer Pattern Mining (CTPM), which automatically clusters to mine hidden transfer patterns and combines contrastive learning to enhance sentence representations, thereby achieving better style control and content preservation.
The Art of Prompting: Event Detection based on Type Specific Prompts
Sijia Wang (Virginia Tech), Lifu Huang (Virginia Tech)
CodeTransformerPrompt EngineeringTextBenchmark
π― What it does: This paper proposes a unified framework that utilizes multiple prompts for event types (naming, definition, seed triggers, structure, soft prompts, and their comprehensive description APEX) for supervised, few-shot, and zero-shot event detection, and verifies its effectiveness through experiments.
The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers
Ariel Gera (IBM Research), Eyal Shnarch (IBM Research)
CodeGenerationTransformerLarge Language ModelContrastive LearningText
π― What it does: Propose Auto-Contrastive Decoding (ACD), which enhances text generation quality by realigning the probability distribution of the next token through contrasting low-level 'amateur' predictions with high-level 'expert' predictions within the same transformer model.
The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources
Akshatha Arodi (McGill University), Jackie Chi Kit Cheung (Microsoft Research)
CodeRepresentation LearningRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: Designed and constructed the KITMUS coreference evaluation dataset to investigate the model's reasoning ability under knowledge source integration during pre-training and inference.
π― What it does: The study investigates the role of local and global context in named entity recognition (NER), proposes and evaluates multiple sentence retrieval heuristics and oracle re-ranking methods, and improves a literary text NER dataset;
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks
Nikil Selvam, Kai-Wei Chang (University of California, Los Angeles)
CodeData-Centric LearningTransformerTextBenchmark
π― What it does: Investigate the reliability of social bias benchmarks by introducing shallow data construction variations on WINOGENDER and BIASNLI, and assess their impact on model bias metrics.
π― What it does: Proposed the Vertical Learning Paradigm (VLP), which enhances Knowledge Graph Completion (KGC) performance by explicitly copying relevant fact triplets, and designed a negative sampling method based on relative distance (ReD) for more effective optimization.
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: This paper proposes Adaptive Prefix Tuning (APT), which dynamically adjusts prefix vectors by using fine-grained token-level gating and coarse-grained hierarchical scaling gating within Transformer layers to enhance the effectiveness of parameter-efficient fine-tuning.
Towards Boosting the Open-Domain Chatbot with Human Feedback
Hua Lu (Baidu), Haifeng Wang (Baidu)
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
π― What it does: Construct the Diamante Chinese casual chat dataset by collecting human selections/rewrites of model-generated candidate responses, and improve open-domain chatbots through generation-evaluation joint training based on this dataset.
π― What it does: Studying how to utilize domain adaptation techniques to improve the accuracy of dementia detection in spoken languages across different domains.
π― What it does: This paper proposes a lightweight supervised open-world product attribute mining framework called Amacer, which can expand existing attribute types and automatically discover new attribute types with only a small number of seed attribute values.
π― What it does: When fine-tuning pre-trained language models (PLM), randomly initialized autoencoders (AE) are inserted between different layers. During forward propagation, these AEs are randomly selected and used to compress hidden representations into multi-view compressed representations (MVCR). During inference, these AEs are removed, keeping the original model's parameters and inference cost unchanged.
Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation
Soyoung Yoon (University of California, Santa Barbara), Alice Oh (KAIST)
CodeData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Collected three Korean grammar error correction (GEC) parallel corpora (Kor-Lang8, Kor-Native, Kor-Learner), and developed an automatic error type annotation tool called KAGAS; subsequently, fine-tuned KoBART to build a baseline model.
Trading Syntax Trees for Wordpieces: Target-oriented Opinion Words Extraction with Wordpieces and Aspect Enhancement
Samuel Mensah (University of Sheffield), Nikolaos Aletras (University of Sheffield)
CodeRecognitionTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper addresses the task of target sentiment word extraction by removing the syntax tree + GCN from the model and instead using BERT subwords (wordpiece) and sentence-target pairs as input, improving aspect representation and achieving precise extraction of target sentiment words.
π― What it does: In the rare category (cognitive dissonance) detection task, combine transfer learning and active learning to improve model performance.
Young Min Kim, David R. Mortensen (Carnegie Mellon University)
CodeGenerationTransformerText
π― What it does: Proposed a prototype word reconstruction model based on Transformer for inferring ancestral word forms from sub-language speech or spelling.
Yaoyiran Li (University of Cambridge), Anna Korhonen (University of Cambridge)
CodeGenerationVision Language ModelGenerative Adversarial NetworkContrastive LearningMultimodality
π― What it does: This paper studies multilingual text-to-image generation (mTTI), aiming to improve image generation quality for non-English languages through machine translation techniques.
Trigger Warning Assignment as a Multi-Label Document Classification Problem
Matti Wiegmann (Bauhaus-UniversitΓ€t Weimar), Martin Potthast (Leipzig University)
CodeClassificationTransformerTextBenchmark
π― What it does: Treat trigger warning assignment as a multi-label text classification task, constructing the Webis Trigger Warning Corpus 2022, which contains approximately 1.09 M AO3 fanfiction works and provides corresponding 36 classes of trigger warning labels.
TwistList: Resources and Baselines for Tongue Twister Generation
Tyler Loakman (University of Sheffield), Chenghua Lin (University of Sheffield)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Proposed the stuttering sentence generation task, released the TwistList dataset (2.1K+ stuttering sentences + keywords + phonetic symbols), and provided a series of baseline models (TwisterMisters).
π― What it does: Investigated the 'overthinking' phenomenon in OOD intent classification and proposed a dynamic early-exit strategy to simultaneously improve speed and accuracy.
π― What it does: This paper proposes using the optimal transport (BOT, POT, UOT) framework to achieve monolingual unbalanced word alignment, utilizing general cost and allocation for alignment in unsupervised scenarios, and implementing end-to-end training through linear metric learning in supervised scenarios.
π― What it does: Developed a label denoising framework called UGDRE based on uncertainty guidance to improve noisy data in document-level distant relation extraction.
Understanding and Improving the Robustness of Terminology Constraints in Neural Machine Translation
Huaao Zhang (RoyalFlush AI Research Institute), Ming Chen (Zhejiang University)
CodeGenerationTransformerTextBenchmark
π― What it does: This study investigates the robustness of terminology-constrained methods in neural machine translation (Placeholder PH and Code-Switch CS), finding inconsistent performance based on the quantity and length of constraints; based on this, a Robust Terminology Translation (RTT) method combining the advantages of PH and CS is proposed, along with the creation of a more challenging English-German terminology-constrained test set.
Understanding Client Reactions in Online Mental Health Counseling
Anqi Li (Zhejiang University), Zhenzhong Lan (Westlake University)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Constructed a theory-driven bidirectional annotation framework tailored for online psychological counseling, encompassing counselor intent, strategies, client reactions, and behaviors; annotated 2,382 Chinese text counseling sessions under this framework; conducted quantitative analysis to examine the impact of client reactions on session effectiveness and how counselors adjust strategies based on feedback; trained a multi-task classification model based on RoBERTa-large, achieving automatic annotation of strategies and reactions;
UniEvent: Unified Generative Model with Multi-Dimensional Prefix for Zero-Shot Event-Relational Reasoning
Zhengwei Tao (Peking University), Chongyang Tao (Microsoft)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringContrastive LearningText
π― What it does: Propose a unified generative model called UNIEVENT, which can perform multiple event relation reasoning tasks in zero-shot scenarios;
Unified Demonstration Retriever for In-Context Learning
Xiaonan Li (Fudan University), Xipeng Qiu (Fudan University)
CodeRetrievalTransformerTextBiomedical DataFinance Related
π― What it does: Propose a Unified Demonstration Retriever (UDR) that can retrieve high-quality demonstrations for a large number of different NLP tasks by using list ranking based on language model feedback and iterative candidate mining on a multi-task bi-encoder.
CodeGenerationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Proposed a unified few-shot text summarization framework called UNISUMM, and released the SUMMZOO benchmark containing 8 diverse tasks; achieved efficient adaptation to any few-shot summarization task through multi-task pre-training and prefix tuning.
UniTRec: A Unified Text-to-Text Transformer and Joint Contrastive Learning Framework for Text-based Recommendation
Zhiming Mao (Chinese University of Hong Kong), Kam-Fai Wong (Chinese University of Hong Kong)
CodeRecommendation SystemTransformerLarge Language ModelContrastive LearningText
π― What it does: Implemented a unified text-to-text Transformer framework named UniTRec, which uses local and global attention for dual-layer encoding of multi-round user history, and combines a pre-trained decoder to calculate the perplexity of candidate text as a matching signal to complete the text recommendation task.
π― What it does: Investigated unsupervised discontinuous constituent analysis using mildly context-sensitive grammar, adopting a probabilistic linear context-free rewriting system (LCFRS) formulation, fixing the rule structure in advance, and focusing on maximum likelihood parameter learning.
Lam Thanh Do (Hanoi University of Science and Technology), Kevin Chen-Chuan Chang (University of Illinois at Urbana-Champaign)
CodeGenerationTransformerMixture of ExpertsTextBenchmarkRetrieval-Augmented Generation
π― What it does: Built an unsupervised, cross-domain keyphrase generation model that utilizes a retrieval-enhanced seq2seq framework and automatically generates core concept phrases through information assessment.
Using Neural Machine Translation for Generating Diverse Challenging Exercises for Language Learner
Frank Palma Gomez (CUNY), Alla Rozovskaya (CUNY)
CodeGenerationTransformerLarge Language ModelText
π― What it does: Generate diverse and challenging fill-in-the-blank distractors for English learners automatically using back-translation neural machine translation technology.
UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction
Hang Yan (Fudan University), Xipeng Qiu (Fudan University)
CodeClassificationTransformerLarge Language ModelText
π― What it does: This paper proposes a unified token-pair classification architecture called UTC-IE, which converts information extraction tasks such as named entity recognition (NER), relation extraction (RE), and event extraction (EE) into a token-pair classification problem.
CodeClassificationRecognitionDomain AdaptationKnowledge DistillationRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose the VendorLink system, which identifies and links sellers migration and potential aliases in dark web markets through NLP writing style recognition, divided into three tasks: closed-set verification, open-set identification, and low-resource market adaptation;
CodeExplainability and InterpretabilityTransformerTextRetrieval-Augmented GenerationChain-of-Thought
π― What it does: Propose a Verify-and-Edit framework that performs post-editing on the generated reasoning chain during the Chain-of-Thought (CoT) inference process to improve the factual accuracy and correctness of answers.
Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Information
Sunjae Kwon (University of Massachusetts Amherst), Hong Yu (University of Massachusetts Lowell)
CodeRetrievalRepresentation LearningLarge Language ModelVision Language ModelMultimodalityRetrieval-Augmented Generation
π― What it does: This paper proposes an unsupervised visual word sense disambiguation method that enhances image-text matching models through Bayesian inference using dictionary definitions.
Visually-augmented pretrained language models for NLP tasks without images
Hangyu Guo (Harbin Institute of Technology (Shenzhen)), Ji-Rong Wen (Renmin University of China)
CodeRepresentation LearningTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelText
π― What it does: Propose a visual augmentation fine-tuning method called VAWI, which does not require image retrieval or generation. It first automatically identifies visual hunger words in text through three strategies (syntax, attention, and learnable), then generates visually aligned representations using the CLIP text encoder. These representations are refined through a position-aware reconstruction layer to obtain visually enhanced representations, which are finally injected into PLMs (supporting both full-parameter or parameter-efficient fine-tuning) to improve performance across various NLP tasks.
VLN-Trans: Translator for the Vision and Language Navigation Agent
Yue Zhang (Michigan State University), Parisa Kordjamshidi (Michigan State University)
CodeRecurrent Neural NetworkTransformerSupervised Fine-TuningReinforcement LearningVision Language ModelVision-Language-Action ModelContrastive LearningImageTextMultimodality
π― What it does: Propose a translation module that converts complete instructions into recognizable and discriminative sub-instructions to enhance visual language navigation performance.
π― What it does: Proposed a word-level alignment contrastive learning method called WACO, which directly brings speech and corresponding text word representations closer, enhancing speech translation performance under extremely low-resource conditions.
We Understand Elliptical Sentences, and Language Models should Too: A New Dataset for Studying Ellipsis and its Interaction with Thematic Fit
Davide Testa (University of Pisa), Alessandro Lenci (University of Pisa)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Studied the ability of language models (GPT-2 and BERT) to process elliptical sentences and explored the impact of typicality of event participants (topic adaptation) on the parsing of elliptical sentences.
π― What it does: Studied the weakly supervised audio-visual localization task and proposed the SIL framework to achieve audio-visual localization without time labels.
π― What it does: This paper introduces the WEBIE dataset, which automatically extracts 1.6M sentences from Common Crawl web text and incorporates negative samples, providing 21K triplets with multilingual translations based on human annotations, aiming to enhance the robustness and generalization capability of closed information extraction.
What does a Text Classifier Learn about Morality? An Explainable Method for Cross-Domain Comparison of Moral Rhetoric
Enrico Liscio (Tudelft), Pradeep K. Murukannaiah (Tudelft)
CodeClassificationDomain AdaptationExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: Proposes the Tomea method, which uses the SHAP interpreter to generate domain-specific moral lexicons, and compares how text classifiers represent moral discourse across different domains.
What does the Failure to Reason with βRespectivelyβ in Zero/Few-Shot Settings Tell Us about Language Models?
Ruixiang Cui (University of Copenhagen), Anders SΓΈgaard (University of Copenhagen)
CodeClassificationData SynthesisExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: This paper systematically investigates the reasoning ability of language models in zero/one-shot scenarios for the 'respectively' structure, constructs synthetic dataset WikiResNLI and natural dataset NatResNLI, and evaluates the performance of multiple Transformer language models on explicit vs. implicit, synthetic vs. natural corpora.
π― What it does: This paper proposes an event argument extraction model APE based on cross-dataset knowledge transfer. It first learns overlapping knowledge across datasets through a pseudo entity recognition (PER) task, then uses a dedicated Adapter to learn dataset-specific knowledge, and activates overlapping knowledge in both stages of training using prompts of the same style.
What social attitudes about gender does BERT encode? Leveraging insights from psycholinguistics
Julia Watson (University of Toronto), Suzanne Stevenson (University of Toronto)
CodeRepresentation LearningTransformerLarge Language ModelText
π― What it does: Evaluate BERT's social attitudes in gender-related language choices by directly linking language selections from human psycholinguistic experiments and social attitude questionnaire results with model predictions.
CodeExplainability and InterpretabilityTransformerDiffusion modelImageText
π― What it does: This paper proposes the DAAM method to generate word-pixel attribution maps by aggregating cross-attention weights from Stable Diffusion and performing bilinear interpolation and thresholding.
When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP
Jingwei Ni (ETH ZΓΌrich), Markus Leippold (University of ZΓΌrich)
CodeClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextFinance Related
π― What it does: This paper uses financial NLP as a case study to systematically evaluate the effectiveness of multi-task learning (MTL) in aggregating diverse skills, and proposes a parameter-efficient SPAL-FinBERT architecture;