These 412 ACL 2023 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every ACL 2023 paper, free trial on arXivSub.
A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairment
Jiyue Jiang (University of Hong Kong), Chuan Wu (University of Hong Kong)
CodeGenerationTransformerLarge Language ModelTextAlzheimer's DiseaseRetrieval-Augmented Generation
π― What it does: Constructed the Chinese Cognitive Stimulation (CS) dialogue dataset CSConv, and proposed a multi-source knowledge fusion CS dialogue system (CSD), achieving emotion support and treatment principle integrated dialogue for elderly cognitive rehabilitation;
A Critical Evaluation of Evaluations for Long-form Question Answering
Fangyuan Xu (University of Texas at Austin), Eunsol Choi (University of Texas at Austin)
CodeTransformerLarge Language ModelTextReview/Survey Paper
π― What it does: This paper conducts a systematic study on evaluation methods for long-form question answering (LFQA), including expert-level human evaluation and reasoning generation, as well as assessing the correlation between various automatically generated metrics and human preferences.
π― What it does: This study constructs a free resource for Turkish NLP, including various corpora (named entity recognition, sentiment analysis, health-related comments, COVID-19 symptoms, etc.), the first Turkish pre-trained model using spaCy (containing lemmatization, morphological analysis, POS, dependency parsing, NER, etc.), as well as a series of video and code tutorials to help users quickly get started and train models.
π― What it does: Designed a two-stage framework called FacialMMT, first extracting accurate facial sequences of real speakers through multimodal face recognition, unsupervised clustering, and face matching, then enhancing multimodal emotion recognition with frame-level expression recognition.
A fine-grained comparison of pragmatic language understanding in humans and language models
Jennifer Hu (Massachusetts Institute of Technology), Edward Gibson (Massachusetts Institute of Technology)
CodeClassificationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Conduct a fine-grained comparison of seven rhetorical phenomena (lies, indirect speech, irony, conversational maxims, metaphor, humor, coherence), using zero-shot prompting to evaluate the performance of multiple language models and humans on these tasks, and analyze error patterns and linguistic cues.
Yutao Sun (Tsinghua University), Furu Wei (Microsoft)
CodeTransformerLarge Language ModelTextSequential
π― What it does: Propose a Transformer model (LEX Transformer) capable of training on short texts and reasoning on long texts, achieving length extrapolation through improved position encoding and attention masks.
A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spaces
Gabriella Chronis (University of Texas at Austin), Katrin Erk (University of Texas at Austin)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: Automatically analyze the semantic construction of vocabulary in different grammatical structures using large language models and an interpretable psycholinguistic feature space.
Clara Meister (ETH ZΓΌrich), Adhiguna Kuncoro (DeepMind)
CodeGenerationTransformerText
π― What it does: In the final linear layer of Transformer language generation models, initializing the bias as the log-unigram distribution of the training corpus provides the model with a natural prior, improving learning efficiency and performance.
A New Dataset and Empirical Study for Sentence Simplification in Chinese
Shiping Yang (Peking University), Xiaojun Wan (Peking University)
CodeGenerationTransformerLarge Language ModelTextBenchmark
π― What it does: This paper creates the Chinese Sentence Simplification Evaluation dataset CSS, conducts data analysis, experiments with unsupervised, zero/few-shot methods, and evaluates large language models;
π― What it does: Proposed a task of simultaneously extracting targets and stances from social media text, along with a two-phase implementation framework.
A Novel Table-to-Graph Generation Approach for Document-Level Joint Entity and Relation Extraction
Ruoyu Zhang (Wangxuan Institute of Computer Technology, Peking University), Lei Zou (TopGraph.AI)
CodeRecognitionGenerationGraph Neural NetworkTransformerLarge Language ModelTextGraph
π― What it does: Proposed an end-to-end model TAG based on table filling to graph generation, jointly performing document-level entity recognition and relation extraction to address the error propagation problem in traditional pipeline models.
π― What it does: Propose a probabilistic framework that optimizes intent allocation using the EM algorithm, leveraging prior label knowledge and contrastive learning to explore the internal structure of unlabeled data during the process.
A Simple and Flexible Modeling for Mental Disorder Detection by Learning from Clinical Questionnaires
Hoyun Song (Korea Advanced Institute of Science and Technology), Jong Park
CodeClassificationTransformerLarge Language ModelContrastive LearningText
π― What it does: Construct and evaluate a Multi-Head Siamese Network (MHS) that directly learns and detects mental disorders from text using DSM-5 symptom descriptions.
π― What it does: This paper reviews efficiency research in open-domain question answering (ODQA), categorizing three major frameworks: Retriever-Reader, Retriever-Only, and Generator-Only. It analyzes their trade-offs between memory usage, query speed, and answer quality, and provides quantitative comparisons and summaries of core technologies.
A Survey of Deep Learning for Mathematical Reasoning
Pan Lu, Kai-Wei Chang (Ucla)
CodeTransformerLarge Language ModelPrompt EngineeringTextMultimodalityReview/Survey PaperBenchmark
π― What it does: This paper reviews the main tasks, datasets, models, and methods of deep learning in mathematical reasoning over the past decade, and proposes systematic classification and future research directions.
A Survey on Asking Clarification Questions Datasets in Conversational Systems
Hossein A. Rahmani (University College London), Aldo Lipani (University College London)
CodeTransformerTextReview/Survey PaperBenchmark
π― What it does: Provide a comprehensive review of research on clarification questions (ACQ) in conversational systems, systematically organize and compare 77 existing papers and their used public datasets, conduct systematic analysis of evaluation metrics, experimental setups, and model performance, and provide visualization methods and benchmark experiment code.
Longyue Wang (Tencent AI Lab), Zhaopeng Tu (Tencent AI Lab)
CodeContrastive LearningTextReview/Survey Paper
π― What it does: This paper reviews the research progress in Zero Pronoun Translation (ZPT), systematically organizing the task evolution, datasets, methods, and evaluation metrics, and proposes key findings and future directions of existing studies.
A Synthetic Data Generation Framework for Grounded Dialogues
Jianzhu Bao (Harbin Institute of Technology), Ruifeng Xu (Huawei Noah's Ark Lab)
CodeData SynthesisTransformerLarge Language ModelText
π― What it does: By constructing dialogue flows from knowledge and using large-scale pre-trained models to progressively generate dialogues, automatically synthesize grounded dialogue data for training.
A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training
Nitay Calderon (Technion - IIT), Amir Kantor (Microsoft)
CodeGenerationKnowledge DistillationText
π― What it does: Systematically study the application of knowledge distillation in natural language generation tasks, propose diverse pseudo-target generation and joint teaching methods, and achieve high compression rates and performance improvements in real-world scenarios with limited annotations.
A Universal Discriminator for Zero-Shot Generalization
Haike Xu (Tsinghua University), Zhilin Yang (Tsinghua University)
CodeClassificationGenerationTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextBenchmark
π― What it does: Trained and evaluated a Universal Discriminator (UD) to determine whether text comes from a real distribution under zero-shot settings, unifying multiple discriminative NLP tasks under this problem.
A Weakly Supervised Classifier and Dataset of White Supremacist Language
Michael Miller Yoder (Carnegie Mellon University), Kathleen M. Carley (Carnegie Mellon University)
CodeClassificationTransformerLarge Language ModelText
π― What it does: Built a weakly supervised framework for detecting white supremacist language, using DistilBERT to perform binary classification on large-scale white supremacist corpora and neutral/anti-racist corpora;
Abstractive Summarizers are Excellent Extractive Summarizers
Daniel Varab (Novo Nordisk IT University of Copenhagen), Yumo Xu (University of Edinburgh)
CodeGenerationTransformerLarge Language ModelText
π― What it does: This paper proposes using a pre-trained generative summarization model (e.g., BART) to directly perform extractive summarization during inference, designing three novel inference algorithmsβgenerative ranking, generative re-ranking, and generative searchβto explore the feasibility and advantages of generative models in extractive summarization.
π― What it does: Propose ACCENT, a reference-free automatic evaluation metric based on event-relation tuples, to measure the event commonsense proficiency of open-domain dialogue systems.
ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER
Sreyan Ghosh (University of Maryland), Dinesh Manocha (University of Maryland)
CodeRecognitionData SynthesisTransformerLarge Language ModelText
π― What it does: Propose ACLM, a generative data augmentation framework based on attention-guided selective masking and text reconstruction, specifically designed for low-resource complex named entity recognition (NER).
π― What it does: Proposes the ACTC method for cold start threshold calibration in knowledge graph completion (KGC), which achieves threshold optimization for each relation through active sample selection and automatic label generation with minimal human annotations (only 10 samples).
π― What it does: Propose an attention-based attribution knowledge distillation framework (AD-KD), which extracts the Integrated Gradients (IG) attribution distribution from the teacher model and transfers it to the student model, enabling the learning of token importance at the input level.
Adaptive and Personalized Exercise Generation for Online Language Learning
Peng Cui (ETH ZΓΌrich), Mrinmaya Sachan (ETH ZΓΌrich)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningTextSequential
π― What it does: This paper proposes a model that combines Knowledge Tracing (KT) with controllable text generation to generate adaptive and personalized language learning exercise sentences.
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkChain-of-Thought
π― What it does: This paper proposes ALERT, a new benchmark that systematically evaluates the performance of large language models on 10 fine-grained reasoning skills (such as logic, causality, common sense, text entailment, mathematics, induction, analogy, inference, spatial reasoning, and argumentation), and uses this benchmark to deeply investigate the impact of reasoning ability during three stages: pre-training, regular fine-tuning, and chain-of-thought (CoT) fine-tuning.
AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model
I-Hung Hsu (University of Southern California), Nanyun Peng (University of California, Los Angeles)
CodeGenerationTransformerPrompt EngineeringText
π― What it does: Proposes AMPERE, a generative event argument extraction model that injects abstract meaning representations (AMR) through hierarchical prefix injection, and introduces an adjusted copying mechanism.
AMR-based Network for Aspect-based Sentiment Analysis
Fukun Ma (Tsinghua University), Lijie Wen (Tsinghua University)
CodeClassificationRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelText
π― What it does: Proposed a Path Aggregation Relation Network (APARN) based on Abstract Meaning Representation (AMR) for Fine-grained Sentiment Analysis (ABSA)
An AMR-based Link Prediction Approach for Document-level Event Argument Extraction
Yuqing Yang (Fudan University), Zheng Zhang (Amazon)
CodeGraph Neural NetworkTransformerTextGraph
π― What it does: Reformulate the document-level event argument extraction problem as link prediction on a customized AMR graph, directly leveraging the graph structure to identify relationships between triggers and arguments.
An Embarrassingly Easy but Strong Baseline for Nested Named Entity Recognition
Hang Yan (Fudan University), Xipeng Qiu (Fudan University)
CodeRecognitionConvolutional Neural NetworkTransformerBiomedical Data
π― What it does: This study proposes stacking a CNN on the score matrix of span-based NER models to capture spatial correlations between neighboring spans, thereby enhancing performance in nested named entity recognition.
An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation
Xuancheng Huang (Meituan), Yang Liu (Tsinghua University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Proposed the Prompt Gating method, achieving a scalable plug-and-play mechanism for multi-dimensional controllable text generation through trainable gate plugins
Guy Dar (Tel Aviv University), Jonathan Berant (Tel Aviv University)
CodeExplainability and InterpretabilityRepresentation LearningTransformerText
π― What it does: This paper proposes a zero-channel method for interpreting Transformer parameters in the embedding space, which projects all weights (keys, values, attention matrices, etc.) using the embedding matrix and analyzes their semantics and functions in this space;
Annotation-Inspired Implicit Discourse Relation Classification with Auxiliary Discourse Connective Generation
Wei Liu (Heidelberg Institute for Theoretical Studies gGmbH), Michael Strube (Heidelberg Institute for Theoretical Studies gGmbH)
CodeClassificationTransformerLarge Language ModelText
π― What it does: Propose an end-to-end neural model that first generates implicit textual connectors between texts, then uses the generated connectors along with arguments to predict implicit discourse relations.
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning
Soumya Sanyal (University of Southern California), Xiang Ren (University of Southern California)
CodeRepresentation LearningTransformerLarge Language ModelText
π― What it does: By filtering sentences from Wikipedia based on logical reasoning keywords and using a self-supervised selective masked language model with sentence classification loss, the pre-trained language model is continuously pre-trained to enhance its logical reasoning ability.
π― What it does: Conduct model integration experiments on Chinese grammar error correction tasks, attempting to select the best output using perplexity calculated by pre-trained language models.
Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark
Wenjun Peng (University of Science and Technology of China), Xing Xie (Microsoft Research Asia)
CodeSafty and PrivacyAdversarial AttackTransformerLarge Language ModelText
π― What it does: Proposed an EmbMarker method based on backdoors to implant inheritable watermarks in Embedding-as-a-Service (EaaS), aiming to protect the copyrights of large language models (LLMs).
Sougata Saha (State University of New York at Buffalo), Rohini Srihari (State University of New York at Buffalo)
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Proposed ArgU, a controllable factual argument generator capable of generating diverse argument texts based on given stances and Walton argument schemes, and released the largest corpus with 69,428 annotated arguments including argument schemes and fact spans.
π― What it does: Proposed a novel adaptive decision strategy called EDATT, which determines when to output partial translations or wait for more audio input by observing encoder-decoder attention weights in offline-trained speech translation models.
π― What it does: Unify the attribute extraction task as a tree generation problem, and propose AtTGen, a text-to-tree generation model, to achieve joint attribute extraction in closed-domain, open-domain, and semi-open-domain scenarios.
Attractive Storyteller: Stylized Visual Storytelling with Unpaired Text
Dingyi Yang (Renmin University of China), Qin Jin (Renmin University of China)
CodeGenerationTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodalityRetrieval-Augmented Generation
π― What it does: Proposes the stylized visual story generation task (SVST) for unpaired text, and designs a multi-task memory-enhanced framework called StyleVSG. It jointly trains on real image sequences and unpaired style text to generate coherent stories with specific writing styles.
Tom Hosking (University of Edinburgh), Mirella Lapata (University of Edinburgh)
CodeGenerationExplainability and InterpretabilityTransformerAuto EncoderText
π― What it does: Propose an unsupervised opinion summarization method that encodes review sentences into a hierarchical discrete latent space, identifies common opinions based on the frequency of latent codes, and generates interpretable abstract and extractive summaries based on these frequent subpaths.
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Zichun Yu (Tsinghua University), Zhiyuan Liu (Tsinghua University)
CodeRetrievalDomain AdaptationTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Proposed a general-purpose retrieval plugin called Augmentation-Adapted Retriever (AAR), which enhances the zero-shot generalization of large language models by training the retriever using attention signals from a small source model without fine-tuning the target language model.
π― What it does: This paper proposes a human evaluation dataset to assess the quality of medical multi-document summarization (MDS) and analyzes the gap between existing automatic evaluation metrics and human assessments.
Back to Patterns: Efficient Japanese Morphological Analysis with Feature-Sequence Trie
Naoki Yoshinaga (University of Tokyo)
CodeComputational EfficiencyText
π― What it does: This paper proposes a greedy longest matching algorithm that utilizes a dictionary and annotated data to extract POS-enhanced patterns, which are stored in a double array trie, achieving an efficient morphological analyzer for Japanese tokenization, part-of-speech tagging, and lemma generation;
Weisong Sun (State Key Laboratory for Novel Software Technology Nanjing University), Bin Luo (State Key Laboratory for Novel Software Technology Nanjing University)
CodeRetrievalAdversarial AttackTransformerText
π― What it does: Researched and implemented a backdoor injection attack on neural code search models, exploiting minor changes in function/variable names to elevate the ranking of vulnerable code snippets in search results.
Balancing Lexical and Semantic Quality in Abstractive Summarization
Jeewoo Sul (Hanyang University), Yong Suk Choi (Hanyang University)
CodeGenerationTransformerContrastive LearningText
π― What it does: Propose the BalSum model in the abstract summary, which adopts a two-stage re-ranking framework to balance lexical overlap and semantic similarity in order to alleviate exposure bias;
Terne Sasha Thorn Jakobsen (Copenhagen Center for Social Data Science), Anders SΓΈgaard (University of Copenhagen)
CodeExplainability and InterpretabilityTransformerSupervised Fine-TuningText
π― What it does: This paper first constructs three interpretable annotated datasets with demographic information (DynaSent, SST-2, CoS-E), and uses these datasets to evaluate the alignment between Transformer model predictions and human reasoning (rationales), exploring model fairness across different age and ethnic groups.
BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval
Shicheng Xu (Institute of Computing Technology, Chinese Academy of Sciences), Xueqi Cheng (Institute of Computing Technology, Chinese Academy of Sciences)
π― What it does: Propose a two-stage beam search to generate monotonic pseudo targets and train a simultaneous translation model using sequence-level knowledge distillation;
π― What it does: Proposed bgGLUE, the first general language understanding evaluation benchmark for Bulgarian, containing nine tasks covering annotation, classification, reasoning, sentiment analysis, and question answering;
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: This paper proposes the Bi-Phone method, which generates spelling errors influenced by L1-L2 by mining phoneme confusions across different language pairs, and synthesizes L2 text based on this; meanwhile, it constructs the FunGLUE benchmark to evaluate model robustness under phoneme errors.
CodeData SynthesisDomain AdaptationTransformerLarge Language ModelText
π― What it does: This paper studies cross-domain aspect-based sentiment analysis (ABSA) and proposes a bidirectional generation framework called BGCA. The framework first trains a text-to-label model using annotated data from the source domain, then generates pseudo-labels on unlabeled data from the target domain. These pseudo-labels are then fed into a label-to-text model to generate natural sentences that align with the labels, producing high-quality pseudo-data. Finally, the source domain data and generated data are jointly retrained in the text-to-label direction to complete cross-domain knowledge transfer.
π― What it does: This study constructs the BIG-C dataset, which includes image-based multi-turn dialogues in the Bemba language, corresponding audio transcriptions, and English translations, covering 92,117 sentences and approximately 180 hours of audio; meanwhile, it provides benchmark experiments for ASR, machine translation, and speech translation.
BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training
Yiming Yan (Nanjing University), Mingxuan Wang (ByteDance AI Lab)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Conduct a systematic analysis of mainstream and state-of-the-art automatic translation evaluation metrics through Minimum Risk Training (MRT), revealing that BLEURT and BARTScore suffer from general adversarial translation defects, and propose using token-level constraints or metric integration to enhance metric robustness and translation quality.
BREAK: Breaking the Dialogue State Tracking Barrier with Beam Search and Re-ranking
Seungpil Won (LG AI Research), Kyomin Jung (Seoul National University)
CodeTransformerLarge Language ModelPrompt EngineeringTextSequential
π― What it does: Propose the BREAK framework, which first generates k dialogue state candidates using beam search, and then selects the most contextually appropriate state using a re-ranker;
Breeding Machine Translations: Evolutionary approach to survive and thrive in the world of automated evaluation
Josef Jon (Charles University), OndΕej Bojar (Charles University)
CodeOptimizationTransformerText
π― What it does: This paper proposes an automated method based on genetic algorithms (GA) and minimum Bayes risk (MBR) decoding to search, improve, and generate new translation candidates from the n-best list produced by machine translation (MT) systems, thereby enhancing translation quality or revealing blind spots in evaluation metrics.
BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering
Jie He (University of Edinburgh), Jeff Pan
CodeClassificationRepresentation LearningTransformerLarge Language ModelContrastive LearningTextGraph
π― What it does: Convert knowledge graph triplets into positive and negative question-answer pairs, train a binary classification model to evaluate sentenceεηζ§, and use it for unsupervised common sense question answering.
ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models
Jonas Belouadi (Bielefeld University), Steffen Eger (Bielefeld University)
CodeGenerationTransformerLarge Language ModelText
π― What it does: Developed an end-to-end style-conditioned poetry generation model named ByGPT5, capable of generating four-line poems that comply with aesthetic constraints such as meter, foot, and alliteration without requiring manual rules or post-processing.
C-STANCE: A Large Dataset for Chinese Zero-Shot Stance Detection
Chenye Zhao (University of Illinois at Chicago), Cornelia Caragea (University of Illinois at Chicago)
CodeClassificationRecurrent Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Constructed and released the Chinese zero-shot stance detection dataset C-STANCE, containing 48,126 Weibo-target pairs covering two types of targets: noun phrases and propositions; proposed two zero-shot stance detection subtasks: target-level and domain-level; conducted experiments on multiple baseline models, demonstrating that Transformer models outperform RNN models, with RoBERTa achieving 78.5% F1 macro average on subtask A; simultaneously performed cross-lingual zero-shot experiments, showing that the Chinese dataset is more challenging.
π― What it does: Investigate whether language models can apply new entity definitions in reasoning tasks after receiving them, propose the entity knowledge propagation task, and construct two evaluation datasets.
CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoning
Weiqi Wang (Hong Kong University of Science and Technology), Lei Chen (Hong Kong University of Science and Technology)
CodeTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
π― What it does: Propose a semi-supervised framework named CAT that integrates event conceptualization and instantiation to automatically generate context-related abstract common-sense knowledge on large knowledge bases.
Causal Intervention and Counterfactual Reasoning for Multi-modal Fake News Detection
Ziwei Chen (Beijing University of Posts and Telecommunications), Liqiang Nie (Harbin Institute of Technology)
CodeClassificationMultimodality
π― What it does: Propose a de-biasing framework named CCD based on causal intervention and counterfactual reasoning for multi-modal fake news detection.
ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?
Michael Heck (Heinrich Heine University DΓΌsseldorf), Milica Gasic
CodeExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper explores the performance of ChatGPT in zero-shot dialogue state tracking (DST) and analyzes its strengths and limitations.
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
Jiaxu Zhao (Eindhoven University of Technology), Mykola Pechenizkiy (University of Technology Sydney)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: Constructed the Chinese dialogue model bias assessment and mitigation dataset CHBias, evaluated and mitigated gender, sexual orientation, age, and appearance biases in Chinese pre-trained dialogue models.
Clinical Note Owns its Hierarchy: Multi-Level Hypergraph Neural Networks for Patient-Level Representation Learning
Nayeon Kim (Seoul National University), Sun Kim (Seoul National University)
CodeRepresentation LearningGraph Neural NetworkTextElectronic Health Records
π― What it does: Propose a clinical note representation learning method based on a multi-layer hypergraph neural network, utilizing word, note-level, and classification-level information from patients' clinical notes to construct a multi-level hypergraph and perform hierarchical message passing, ultimately achieving prediction of in-hospital mortality rates for ICU patients.
CoAD: Automatic Diagnosis through Symptom and Disease Collaborative Generation
Huimin Wang (Tencent), Yefeng Zheng (Tencent)
CodeClassificationTransformerElectronic Health Records
π― What it does: This paper proposes CoAD, an automatic diagnosis framework based on the Transformer decoder, which can collaboratively generate symptom sequences and disease labels to directly realize question-answer style diagnosis.
CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors
Peng Li (Fudan University), Xipeng Qiu (Fudan University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Propose converting named entity recognition and relation extraction tasks into code generation tasks, leveraging large code language models (e.g., Codex) for information extraction in few-shot settings.
π― What it does: Propose a sample selection method called PATRON for few-shot fine-tuning of pre-trained language models in a cold-start scenario with no labeled data available at the beginning.
π― What it does: The study investigates the impact of relaxing entity boundary annotations on the performance of an entity linking system by randomly expanding entity boundaries on the AIDA CoNLL-YAGO dataset to generate noisy data; training is conducted on three models (HPAELDC, LUKE, and a custom VanillaNER+LUKE), and the performance of the NER and ED stages is evaluated on the original test set; average experiments are conducted across different noise ratios, analyzing matching types and vocabulary dependencies.
π― What it does: Propose a contrastive learning approach based on sentence component combination, constructing positive samples by splitting sentences into left and right phrases while retaining the training framework of SimCSE;
Compositional Generalization without Trees using Multiset Tagging and Latent Permutations
Matthias Lindemann (University of Edinburgh), Ivan Titov (University of Edinburgh)
CodeTransformerLarge Language ModelTextBenchmark
π― What it does: Propose a two-stage sequence-to-sequence model: first predict an output multiset for each input word, then arrange the words in order using a differentiable sparse permutation model.
π― What it does: Propose the ConFEDE framework, which enhances the representation capability for video multimodal sentiment analysis by jointly training with contrastive feature decomposition and contrastive learning.
π― What it does: Proposed a consistency regularization training method to enhance the compositional generalization ability of Transformers, particularly in semantic parsing and machine translation tasks.
π― What it does: Propose the N-way K-shot continual relation extraction (NK-CRE) task and design the ConPL method to alleviate catastrophic forgetting and similar class confusion under few-shot conditions.
Constrained Tuple Extraction with Interaction-Aware Network
Xiaojun Xue (Beijing Institute of Technology), Zhendong Niu (Beijing Institute of Technology)
CodeGraph Neural NetworkTransformerLarge Language ModelText
π― What it does: Proposes the Constrained Triplet Extraction (CTE) task and realizes the extraction of knowledge triplets with spatiotemporal and conditional constraints from text through an Interaction-Aware Network (IAN).
Context-Aware Transformer Pre-Training for Answer Sentence Selection
Luca Di Liello (University of Trento), Alessandro Moschitti (Amazon Alexa AI)
CodeRetrievalRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This work designs three pre-training objectives (SSP) that align with answer sentence selection (AS2), conducts continuous pre-training on RoBERTa and ELECTRA, and then fine-tunes on multiple public and industrial AS2 datasets, ultimately enhancing the model's ability to leverage contextual information.
Contextual Knowledge Learning for Dialogue Generation
Wen Zheng (University of Nottingham), Ke Zhou (University of Nottingham)
CodeGenerationTransformerLarge Language ModelText
π― What it does: Propose the Contextual Knowledge Learning (CKL) model to simultaneously perform fine-grained weighting of context and external knowledge in dialogue generation, thereby improving the quality of generated responses.
π― What it does: Designed and implemented CONTRACLM, a framework that simultaneously employs token-level and sequence-level contrastive learning in causal language models to enhance model representation resolution and multi-task performance.
π― What it does: This study addresses the coarse-to-fine hierarchical text classification task, proposing a lightweight contrastive clustering bootstrapping method for iterative refinement of document labels.
CodeExplainability and InterpretabilityKnowledge DistillationData-Centric LearningTransformerContrastive LearningText
π― What it does: This paper proposes a novel contrastive error attribution framework (Contrastive Error Attribution, CEA) to identify and remove low-quality training samples that cause unreliable outputs (such as hallucinations and semantic errors in text summarization) in natural language generation models.
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Propose using prompt methods with large language models as an alternative to traditional fine-tuning to control hybrid dialogue generation.
Controllable Text Generation via Probability Density Estimation in the Latent Space
Yuxuan Gu (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
CodeGenerationFlow-based ModelAuto EncoderText
π― What it does: This paper proposes a reversible transformation framework that uses probability density estimation in the latent space to achieve controllable generation of text attributes;
ConvGQR: Generative Query Reformulation for Conversational Search
Fengran Mo (University of Montreal), Jian-Yun Nie (University of Montreal)
CodeGenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Propose the ConvGQR framework, which leverages a generative pre-trained language model to simultaneously perform query rewriting and potential answer generation, thereby enhancing conversational retrieval effectiveness.
π― What it does: Proposed a Counterfactual Active Learning (CounterAL) framework that combines active learning with counterfactual sample construction to enhance model generalization on out-of-distribution (OOD) tasks.
Counterfactual Multihop QA: A Cause-Effect Approach for Reducing Disconnected Reasoning
Wangzhen Guo (Sun Yat-Sen University), Hanjiang Lai (Sun Yat-Sen University)
CodeTransformerTextChain-of-Thought
π― What it does: Proposed a counterfactual multi-hop question answering method based on causal inference, aiming to reduce disconnection reasoning (i.e., providing correct answers using only a single fact) in multi-hop QA and enhance genuine multi-hop reasoning capabilities.
CodeGenerationExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: Propose the CREST framework, combining sparse rationalization (Selective Rationalization) with counterfactual text generation (Counterfactual Generation), achieving dual improvements in explainability and robustness.
Cross-Domain Data Augmentation with Domain-Adaptive Language Modeling for Aspect-Based Sentiment Analysis
Jianfei Yu (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
CodeClassificationGenerationData SynthesisDomain AdaptationRecurrent Neural NetworkTransformerLarge Language ModelText
π― What it does: Proposes a three-stage cross-domain data augmentation framework DAβ―LM, which first uses source domain labeled data and target domain unlabeled data to generate pseudo-labels, then trains a domain-adaptive language model (DALM) to jointly generate words and BIO labels, and finally generates a large amount of fluent and diverse target domain labeled data in an autoregressive manner for training cross-domain aspect-based sentiment analysis (ABSA) and aspect extraction (AE) models.
Cross-lingual Science Journalism: Select, Simplify and Rewrite Summaries for Non-expert Readers
Mehwish Fatima (Heidelberg Institute for Theoretical Studies), Michael Strube (Heidelberg Institute for Theoretical Studies)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
π― What it does: Propose the cross-lingual scientific journalism (CSJ) task and design a three-stage pipeline SSR (SELECT β SIMPLIFY β REWRITE) to generate localized language (German) popular science abstracts for non-expert readers from English scientific texts.
π― What it does: Propose a multi-perspective capability-based curriculum learning framework (MCCL), which arranges and prioritizes training samples by combining multiple graph complexity metrics with the model's capability dynamics during training;
DarkBERT: A Language Model for the Dark Side of the Internet
Youngjin Jin (KAIST), Seungwon Shin (KAIST)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Constructed and pre-trained a language model specifically for dark web English text called DarkBERT, and evaluated it on multiple dark web-related tasks (activity classification, ransomware leak site detection, forum thread detection, keyword reasoning).
Data Curation Alone Can Stabilize In-context Learning
Ting-Yun Chang (University of Southern California), Robin Jia (University of Southern California)
CodeClassificationData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Studies how to improve the stability and accuracy of large language models in parameter-free few-shot learning by selecting subsets of training data.
DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions
Vijay Viswanathan (Carnegie Mellon University), Graham Neubig (Carnegie Mellon University)
CodeRetrievalRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Proposed a dataset recommendation task based on natural language descriptions and constructed the DataFinder dataset to support training and evaluation.
John Pavlopoulos (Athens University of Economics and Business), Asimina Paparigopoulou (Ca'Foscari University of Venice)
CodeData-Centric LearningText
π― What it does: Constructed and made public a dataset of 389 transcribed Greek documentary papyri texts, and used text regression methods to predict their dates;