🎯 What it does: A generative named entity recognition model based on Seq2Seq, which uses re-ranking techniques to correct the likelihood distribution of candidate sequences, reducing bias during training.
🎯 What it does: In the Model-as-a-Service (MaaS) scenario, the Decoder Tuning (DecT) method is proposed, which constructs a tunable decoder at the output of the pre-trained model (PTM) to achieve efficient adaptation to downstream tasks.
🎯 What it does: Propose a denoising bottleneck fusion (DBF) model that removes redundancy and noise in video multimodal data while preserving key information through restricted receptive field bottleneck embeddings and mutual information maximization (MI-Max), achieving higher quality cross-modal fusion;
Dependency resolution at the syntax-semantics interface: psycholinguistic and computational insights on control dependencies
Iria de-Dios-Flores (Universidade de Santiago de Compostela), Marcos Garcia (Universidade de Santiago de Compostela)
CodeRecognitionTransformerLarge Language ModelText
🎯 What it does: This study compares the ability of humans and pre-trained masked language models to identify dependencies in control structures in Spanish and Galician through psycholinguistic and computational experiments.
🎯 What it does: Proposed the DEPLAIN dataset, containing sentence-level and document-level simplification alignments in German text, and provides both manual and automatic alignment methods;
Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better
David Dale (Meta AI), Marta R. Costa-jussà (Meta AI)
CodeGenerationAnomaly DetectionExplainability and InterpretabilityData-Centric LearningTransformerText
🎯 What it does: The study proposes using the internal ALTI+ method of Transformers to evaluate the contribution of source sentences in translation, aiming to detect and correct hallucinations in machine translation.
Detecting Contradictory COVID-19 Drug Efficacy Claims from Biomedical Literature
Daniel Sosa, Russ Altman
CodeClassificationTransformerSupervised Fine-TuningBiomedical Data
🎯 What it does: This paper proposes a model based on natural language inference (NLI) to automatically identify contradictory statements in studies on the efficacy of drugs for COVID-19.
CodeExplainability and InterpretabilityGraph Neural NetworkTransformerTextBenchmark
🎯 What it does: This paper first creates the DocREDHWE dataset, adding human-annotated word-level evidence to relation facts in DocRED; subsequently, it uses a feature attribution method (Integrated Gradients) to analyze the decision rules of SOTA models in DocRE, discovering that models mainly rely on non-causal vocabulary (e.g., entity names, periods), and introduces the MAP metric to evaluate the similarity between model decisions and human decision rules; then, it verifies model robustness through various word-level attacks (masking, synonym/antonym substitution, entity name masking/shuffling/substitution).
🎯 What it does: This paper creates and publicly releases DIFFUSIONDB—the first large-scale text-to-image prompt database, containing 14 million Stable Diffusion-generated images, 1.8 million unique prompts, and corresponding hyperparameters;
DISCO: Distilling Counterfactuals with Large Language Models
Zeming Chen (EPFL), Kyle Richardson (Allen Institute for AI)
CodeData SynthesisTransformerLarge Language ModelText
🎯 What it does: Propose the DISCO framework, which generates and filters high-quality counterfactual data using large language models for model training
DiSCoMaT: Distantly Supervised Composition Extraction from Tables in Materials Science Articles
Tanishq Gupta (Indian Institute of Technology Delhi), Mausam (Indian Institute of Technology Delhi)
CodeGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextTabularPhysics Related
🎯 What it does: Propose the task of extracting material composition from tables in materials science papers and implement the baseline system DISCOMAT;
🎯 What it does: Propose a document-level event relation extraction model called SENDIR that does not require prior knowledge and is based on sparse attention and distinguishing intra-sentence and inter-sentence reasoning
🎯 What it does: This paper proposes a Chinese spelling error correction model called DORM, which significantly improves the utilization of pinyin information by decoupling text and pinyin representations and enabling bidirectional interaction within the Transformer.
DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering
Ella Neeman (Hebrew University of Jerusalem), Omri Abend (Hebrew University of Jerusalem)
CodeGenerationRetrievalExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Train a single generative QA model to simultaneously provide context-based answers and model-parameter-based answers in one inference, achieving decoupling of knowledge sources.
Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis
Ta-Chung Chi (Carnegie Mellon University), Peter Ramadge
CodeExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper systematically analyzes the length extrapolation capabilities of ALiBi and window attention using an observable cumulative normalized gradient tool, and proposes a new parameter-free relative position encoding called Sandwich, demonstrating its superiority on long sequences.
🎯 What it does: Propose a diversity-aware consistency loss that directly incorporates corpus-level consistency metrics (NPMI) and diversity constraints into the training of neural topic models.
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper systematically studies the differences in literalness between GPT series large language models and traditional neural machine translation, verifying through a combination of quantitative metrics and human evaluation.
Do I have the Knowledge to Answer? Investigating Answerability of Knowledge Base Questions
Mayur Patidar (TCS Research), Mausam (Indian Institute of Technology)
CodeGraphBenchmark
🎯 What it does: In the knowledge graph question answering task, a method for detecting answerability was proposed and evaluated, a benchmark dataset containing unanswerable questions named GrailQAbility was constructed, and the impact of different types of knowledge base incompleteness on models was systematically analyzed.
Document-Level Multi-Event Extraction with Event Proxy Nodes and Hausdorff Distance Minimization
Xinyu Wang (University of Warwick), Yulan He (King's College London)
CodeGraph Neural NetworkTransformerLarge Language ModelTextFinance Related
🎯 What it does: ProCNet proposes using event proxy nodes and minimizing Hausdorff distance to directly learn multi-event extraction at the document level globally, freeing itself from traditional entity relationship modeling and step-by-step decoding processes.
Does GPT-3 Grasp Metaphors? Identifying Metaphor Mappings with Generative Language Models
Lennart Wachowiak (King's College London), Dagmar Gromann (University of Vienna)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper investigates whether GPT-3 can directly predict the source domain of metaphors based on sentences and target domains, proposing an evaluation method based on few-shot prompting and fine-tuning.
🎯 What it does: Propose a dual alignment pre-training framework (DAP), introducing simultaneously a sentence-level translation ranking task and a token-based representation translation learning (RTL) task on dual encoders to achieve better cross-sentence embeddings;
DualGATs: Dual Graph Attention Networks for Emotion Recognition in Conversations
Duzhen Zhang (Baidu Inc), Xiuyi Chen (Baidu Inc)
CodeRecognitionGraph Neural NetworkLarge Language ModelTextGraph
🎯 What it does: This paper proposes the DualGATs model, which uses DisGAT to capture discourse hierarchical structures, SpkGAT to capture speaker dependencies, and combines the two types of information through mutual attention interaction;
Dynamic and Efficient Inference for Text Generation via BERT Family
Xiaobo Liang (Soochow University), Min Zhang (Soochow University)
CodeGenerationTransformerScore-based ModelText
🎯 What it does: Utilize BERT family models for non-autoregressive text generation, achieving efficient inference through joint training of a single-step CTC generator and Levenshtein editor.
Dynamic Routing Transformer Network for Multimodal Sarcasm Detection
Yuan Tian (Institute of Automation, Chinese Academy of Sciences), Wenji Mao (Institute of Automation, Chinese Academy of Sciences)
CodeClassificationTransformerLarge Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a dynamic routing Transformer network, DynRT-Net, to capture cross-modal contradictions between images and text, thereby achieving multimodal sarcasm detection.
Easy Guided Decoding in Providing Suggestions for Interactive Machine Translation
Ke Wang (Alibaba Group Inc), Yu Zhao (Alibaba Group Inc)
CodeGenerationTransformerLarge Language ModelText
🎯 What it does: Proposed an unsupervised decoding algorithm called Prefix-Suffix Guided Decoding (PSGD) for the translation suggestion task in computer-assisted translation, which generates alternative translations for text segments with marked errors under prefix and suffix constraints.
Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Method
Yiming Wang (Shanghai Jiao Tong University), Rui Wang (Shanghai Jiao Tong University)
CodeGenerationTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Constructed an expert-constructed news element-aware test set and evaluated LLMs and traditional pre-trained models on it; proposed a step-by-step summarization generation method based on chain-of-thought (SumCoT).
EM Pre-training for Multi-party Dialogue Response Generation
Yiyang Li (Shanghai Jiao Tong University), Hai Zhao (Shanghai Jiao Tong University)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningTextSequential
🎯 What it does: Pre-train multi-party dialogue response generation using the EM framework, first generating implicit recipient labels via the expectation step, then training the response generation model through the maximization step; no recipient annotations are required during pre-training, but real annotations can be used in subsequent fine-tuning.
Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation
Chen Tang (University of Surrey), Frank Guerin (University of Surrey)
CodeGenerationGraph Neural NetworkTransformerLarge Language ModelTextGraphRetrieval-Augmented Generation
🎯 What it does: Proposed a dialogue generation framework based on dynamic graph knowledge aggregation (SaBART), which constructs pseudo nodes to dynamically generate subgraphs and aggregates knowledge at multiple levels, achieving seamless integration of language models with knowledge graphs.
Jiaxin Ge (Peking University), James Glass (Massachusetts Institute of Technology)
CodeClassificationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper utilizes a text entailment model for pre-training and achieves zero-shot adaptation by converting different NLU tasks into entailment forms (supposition). Subsequently, the authors propose a SimPLE algorithm based on pseudo-label self-training, which generates multi-labels using dropout, filters uncertain samples based on proximity similarity, and re-labels through majority voting to enhance self-training quality.
🎯 What it does: Create EPIC, a multi-perspective irony annotated corpus containing social media dialogues from five English variants, annotated by binary classification from annotators in the corresponding countries.
ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain
Mike Zhang (IT University of Copenhagen), Barbara Plank (IT University of Copenhagen)
CodeDomain AdaptationRepresentation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Proposed and trained a multilingual domain-adaptive pre-trained model ESCOXLM-R based on XLM-R large, utilizing the European Skills and Occupations Taxonomy (ESCO) for domain pre-training and introducing a new relation prediction task;
🎯 What it does: Proposed the Annotated 3D Shapes (A3DS) dataset and used it to evaluate the pragmatic performance of image captioners in reference tasks, comparing captioners fine-tuned with multi-agent reinforcement learning to baseline models.
Erica Cai (University of Massachusetts Amherst), Brendan O’Connor
CodeData-Centric LearningTextBenchmark
🎯 What it does: This paper investigates the issues present when evaluating zero-shot (zero-shot) event extraction methods using the ACE corpus, and proposes three improvement suggestions (coreference matching, dual head selection, and modality/event subset evaluation)
Event Extraction as Question Generation and Answering
Di Lu (Dataminr Inc), Alejandro Jaimes (Dataminr Inc)
CodeGenerationTransformerLarge Language ModelText
🎯 What it does: Propose a context-aware event argument extraction framework QGA-EE based on question generation and answer answering, which directly generates context-aware questions and answers using a seq2seq model.
CodeRecommendation SystemExplainability and InterpretabilityTransformerTextRetrieval-Augmented Generation
🎯 What it does: Proposed a retrieval-augmented and aspect-based explainable recommendation model (ERRA) that simultaneously predicts ratings and generates personalized explanations.
Explaining How Transformers Use Context to Build Predictions
Javier Ferrando (TALP Research Center, Universitat Politècnica de Catalunya), Marta R. Costa-jussà (Meta AI)
CodeExplainability and InterpretabilityTransformerText
🎯 What it does: Propose an interpretability method combining residual flow and attention decomposition to analyze the contribution of context in Transformer language generation models when generating each word.
Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Model
Chantal Amrhein (University of Zurich), Samuel Läubli (University of Zurich)
CodeGenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes using existing gender bias models to generate biased text, then training a neural rewrite model to achieve gender fair language generation.
Exploring and Verbalizing Academic Ideas by Concept Co-occurrence
Yi Xu (Shanghai Jiao Tong University), Chenghu Zhou (IGSNRR Chinese Academy Of Sciences)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Construct evolutionary concept co-occurrence graphs and co-occurrence citation quintuples datasets, and design a temporal link prediction based on masked language models and idea generation based on pre-trained language models, integrated into a real-time research assistant system.
Exploring Continual Learning for Code Generation Models
Prateek Yadav (University of North Carolina Chapel Hill), Bing Xiang (AWS AI Labs)
CodeGenerationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes a continuous learning benchmark called CODETASK-CL for code generation and systematically evaluates the performance of various continuous learning methods on this benchmark.
Exploring Large Language Models for Classical Philology
Frederick Riemenschneider (Heidelberg University), Anette Frank (Heidelberg University)
CodeClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Constructed and evaluated four pre-trained language models for classical languages (Ancient Greek and Latin), including monolingual and multilingual versions of RoBERTa and T5, and conducted systematic benchmarking on tasks such as part-of-speech tagging, dependency syntax, and lemmatization.
Exploring the Capacity of Pretrained Language Models for Reasoning about Actions and Change
Weinan He (Sun Yat-sen University), Yongmei Liu (Sun Yat-sen University)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed and implemented TRAC, a text-based action and change reasoning benchmark, encompassing four tasks: projection, executability, plan verification, and goal recognition;
Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization
Shiyue Zhang (University of North Carolina Chapel Hill), Mohit Bansal (University of North Carolina Chapel Hill)
CodeGenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Analyzed and systematically distilled five common types of unfaithfulness in extractive summarization, constructed a dataset with 1,600 human-annotated examples, and proposed a new evaluation metric called EXTEVAL.
Fact-Checking Complex Claims with Program-Guided Reasoning
Liangming Pan (University of California Santa Barbara), Preslav Nakov (MBZUAI)
CodeRetrievalExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed the Program-Guided Fact-Checking (PROGRAMFC) framework, which leverages large language models to first generate interpretable reasoning programs, then sequentially invokes specialized subtask functions to accomplish multi-step evidence retrieval and fact verification;
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Eve Fleisig (University of California Berkeley), Hanna Wallach (Microsoft Research)
CodeGenerationTransformerLarge Language ModelText
🎯 What it does: Proposed the FairPrism dataset, systematically collected and meticulously annotated 5,000 AI-generated texts for fairness harms related to gender and sexual orientation, covering stereotypes and degrading harms as well as special risks in interactive contexts;
FC-KBQA: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answering
Lingxi Zhang (Renmin University of China), Juanzi Li (Renmin University of China)
CodeTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: By detecting fine-grained components and applying intermediate-grained constraints, then generating executable logical expressions via seq2seq, the problem of generalization and executability in KBQA is addressed.
FERMAT: An Alternative to Accuracy for Numerical Reasoning
Jasivan Sivakumar, Nafise Sadat Moosavi (University of Sheffield)
CodeTextBenchmark
🎯 What it does: This paper proposes FERMAT, a multi-perspective evaluation dataset for fine-grained assessment of language models' numerical reasoning capabilities.
🎯 What it does: This paper conducts a comprehensive empirical study on few-shot event detection (ED) methods, proposing a unified perspective and a better baseline model.
Few-shot In-context Learning on Knowledge Base Question Answering
Tianle Li (University of Waterloo), Wenhu Chen (Vector Institute)
CodeTransformerLarge Language ModelGraphBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose a training-agnostic KBQA framework called KB-BINDER, which utilizes large language models to generate logical form drafts, then binds entities and relations on the knowledge graph, and finally executes to obtain answers.
Finding the Pillars of Strength for Multi-Head Attention
Jinjie Ni (Nanyang Technological University), Erik Cambria (Nanyang Technological University)
CodeComputational EfficiencyTransformerText
🎯 What it does: Design Grouped Head Attention through grouped constraint training (GCT) and voting retention (V2S) to reduce redundancy and over-parameterization in multi-head attention;
🎯 What it does: Conduct a comparative study of adaptive inference methods (Early-Exit and Multi-Model) in low-resource scenarios, and propose an improved algorithm named SWEET to address the gradient conflict problem in Early-Exit
FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information
Andrew Zhu (University of Pennsylvania), Chris Callison-Burch (University of Pennsylvania)
CodeGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Built and utilized a large-scale real D&D game corpus named FIREBALL, providing 25k scenarios, 8M sentences, 2.1M commands, 1.2M game states, to train language models for completing two tasks: 'sentence→command' and 'state→narration'.
🎯 What it does: This paper proposes organizing key points obtained from Key Point Analysis (KPA) hierarchically into Key Point Hierarchies (KPH), and constructs the first KPH benchmark dataset named THINKP; meanwhile, it designs and evaluates multiple methods for predicting hierarchical relationships from key points;
FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction
Tianshuo Peng (Wuhan University), Hai Zhao (Shanghai Jiao Tong University)
CodeTransformerText
🎯 What it does: Proposed a generic information extraction framework FSUIE based on fuzzy boundaries, improving the overfitting issue in span prediction of traditional UIE models
FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue
Weihao Zeng (Beijing University of Posts and Telecommunications), Weiran Xu (Meituan)
CodeKnowledge DistillationRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose the FutureTOD pre-training model, which utilizes a self-training framework to distill future dialogue information into the current dialogue representation.
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
Jian Yang (State Key Lab of Software Development Environment Beihang University), Zhoujun Li (State Key Lab of Software Development Environment Beihang University)
🎯 What it does: Propose a GAN-based Encoder-Decoder pre-training framework called GANLM, which uses an auxiliary discriminator to jointly perform replacement word detection and replacement word denoising tasks, enhancing both generation and understanding capabilities.
Generating Structured Pseudo Labels for Noise-resistant Zero-shot Video Sentence Localization
Minghang Zheng (Peking University), Yang Liu (Peking University)
CodeRetrievalLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Propose a zero-shot video sentence localization framework SPL, which first generates free-form pseudo queries using the BLIP image captioning model, then selects query-dependent event proposals by sliding window and intra/inter-event difference based on the similarity between the query and video frames, followed by non-maximum suppression to filter high-quality pseudo query-event pairs. Subsequently, a noise-robust training method with sample reweighting and pseudo label refinement is employed to train a fully supervised localization model.
Generating Visual Spatial Description via Holistic 3D Scene Understanding
Yu Zhao (Tianjin University), Tat-Seng Chua (National University of Singapore)
CodeGenerationGraph Neural NetworkTransformerPrompt EngineeringVision Language ModelImagePoint Cloud
🎯 What it does: Leverage an external 3D scene extractor to obtain 3D object and scene features from images, constructing a goal-oriented 3D spatial scene graph (GO3D-SG) centered on the target object, and generating visual spatial descriptions (VSD) through this graph;
🎯 What it does: Propose a lightweight graph-induced fine-tuning method called GIFT, which utilizes the reply graph structure of multi-party dialogues to refine attention weights within the Transformer through four types of edge relationships (reply-to, replied-by, reply-self, indirect-reply), thereby enhancing the understanding performance of multi-party dialogues.
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages
Ayyoob Imani (LMU Munich), Hinrich Schütze (LMU Munich)
CodeRepresentation LearningData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Trained a multilingual large model called Glot500-m covering 511 low-resource languages, and constructed the corresponding 600GB corpus Glot500-c
Gradient Ascent Post-training Enhances Language Model Generalization
Dongkeun Yoon (KAIST), Minjoon Seo (KAIST)
CodeClassificationGenerationData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Post-training is performed on the pre-trained OPT language models (350M, 1.3B, 2.7B) by executing a few gradient ascent steps (up to 15 steps) on random unannotated text, referred to as GAP.
🎯 What it does: Proposed a lightweight knowledge graph completion method called GreenKGC, which employs a three-stage independent training process (representation learning, feature pruning, decision learning) to achieve efficient and accurate predictions in low-dimensional spaces.
Grounded Multimodal Named Entity Recognition on Social Media
Jianfei Yu (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
CodeRecognitionObject DetectionTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: This paper proposes a new multimodal named entity recognition task—Grounded Multimodal Named Entity Recognition (GMNER), which requires identifying entities, entity types, and corresponding visual bounding boxes from text-image social media posts; simultaneously constructs a GMNER dataset based on Twitter; and proposes a hierarchical index generation framework, H-Index, which uses BART to perform end-to-end generation of entity-type-region triplets;
Sandeep Soni (University of California, Berkeley), David Bamman (University of California, Berkeley)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Proposed and implemented a multi-classification task for attributing spatial relationships between characters and locations in narrative texts, constructing approximately 2500 annotated samples.
🎯 What it does: This paper proposes a hierarchical attention model HAHE for embedding hyper-relational knowledge graphs, which simultaneously considers the global hypergraph structure and local semantic sequences;
HAUSER: Towards Holistic and Automatic Evaluation of Simile Generation
Qianyu He (Fudan University), Yunwen Chen (DataGrand Inc)
CodeGenerationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed HAUSER——an overall automatic evaluation system for simile generation tasks, along with five evaluation metrics that cover three dimensions: quality, creativity, and informativeness.
HermEs: Interactive Spreadsheet Formula Prediction via Hierarchical Formulet Expansion
Wanrong He (Tsinghua University), Dongmei Zhang (Microsoft Research Asia)
CodeAI Code AssistantTransformerLarge Language ModelTabular
🎯 What it does: Propose HERMES, a framework for predicting spreadsheet formulas through hierarchical formula expansion, and implement an interactive formula completion interface.
🎯 What it does: Propose a hierarchical prompting (HierVerb) method that uses multi-layer learnable verbalizers to embed hierarchical label knowledge into pre-trained language models (PLMs) for hierarchical text classification under few-shot scenarios.
HINT: Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation
Hamish Ivison (Allen Institute for AI), Matthew Peters (Allen Institute for AI)
CodeComputational EfficiencyRepresentation LearningMeta LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose a hypernetwork-based instruction tuning framework called HINT, which can map task instructions and a few examples into parameter-efficient modules (adapter and prefix) in one go, and concatenate the encoded instructions with the input during the decoding stage.
CodeClassificationRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper pre-trains RoBERTa language models on five different morphological language types (English, French, German, Turkish, Quechua) at scales of 1M–6M tokens using three tokenizers (BPE, Unigram, Canonical DeepSpin), and fine-tunes them on two downstream tasks (NER and POS), systematically evaluating the impact of data size and tokenization methods on LM perplexity and downstream performance.
HistRED: A Historical Document-Level Relation Extraction Dataset
Soyoung Yang (KAIST), Jaegul Choo (KAIST)
CodeTransformerTextBenchmark
🎯 What it does: Constructed HistRED—a bilingual (Korean + Chinese characters) relation extraction dataset based on historical documents from the Korean Joseon Dynasty's 'Yeonhaengrok,' and proposed a cross-lingual attention model leveraging bilingual context for document-level relation extraction.
🎯 What it does: Propose the HiTIN model, which utilizes an encoding tree constructed through structural entropy minimization in hierarchical text classification to efficiently inject label hierarchy information into text representations.
🎯 What it does: Propose the AGATE framework, capable of comprehensively predicting the future of temporal evolving attribute graphs, including node attributes, link additions/removals, and the emergence/disappearance of new/old nodes; and implement a novel new node attribute prediction method called PROSER within the framework.
Ryosuke Yamaki (Ritsumeikan University), Daichi Mochihashi (Institute of Statistical Mathematics)
CodeRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose to treat Combinatory Categorial Grammar (CCG) as recursive combinations in a continuous vector space, constructing phrase- and sentence-level representations through recursive HolE embeddings; achieve high-precision super-tagging and parsing based on this; simultaneously utilize decomposability to realize phrase-level text fill-in.
How Do In-Context Examples Affect Compositional Generalization?
Shengnan An (Xi'an Jiaotong University), Dongmei Zhang (Microsoft Corporation)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper investigates how the similarity, diversity, and complexity of examples in the context learning of large language models affect compositional reasoning ability, and proposes the COFE benchmark suite.
🎯 What it does: This study comprehensively evaluates the knowledge distillation targets of the BERT model, with a particular focus on the impact of weight initialization and distillation targets on model compression.
Human-in-the-loop Evaluation for Early Misinformation Detection: A Case Study of COVID-19 Treatments
Ethan Mendes (Georgia Institute of Technology), Alan Ritter (Georgia Institute of Technology)
CodeClassificationAnomaly DetectionTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Constructed a human-machine collaborative early rumor detection evaluation framework and implemented case studies on COVID-19 treatment misinformation.
Hybrid Uncertainty Quantification for Selective Text Classification in Ambiguous Tasks
Artem Vazhentsev (AIRI), Artem Shelmanov (London Institute for Mathematical Sciences)
CodeClassificationTransformerText
🎯 What it does: Propose a hybrid uncertainty quantification method (HUQ), combining the likelihood distribution of prior knowledge (epistemic) with entropy related to softmax (aleatoric), to enhance selective classification performance in text classification tasks with high ambiguity (e.g., toxicity detection).
CodeRecognitionTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper constructs the first Arabic disaster-related tweet location extraction dataset, IDRISI-RA, and provides both gold standard and silver standard versions.
Improving Automatic Quotation Attribution in Literary Novels
Krishnapriya Vishnubhotla (University of Toronto), Adam Hammond (University of Toronto)
CodeClassificationRecognitionRecurrent Neural NetworkTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper modularly decomposes the citation attribution task in literary novels based on the Project Dialogism Novel Corpus, separately evaluating four subtasks: character identification, coreference resolution, quote identification, and speaker attribution, and proposes an improved speaker attribution model on this basis.
Improving Continual Relation Extraction by Distinguishing Analogous Semantics
Wenzheng Zhao (Nanjing University), Wei Hu (Nanjing University)
CodeClassificationKnowledge DistillationRepresentation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Propose a novel continuous relation extraction model to address the forgetting of semantically similar relations and sample overfitting issues.
🎯 What it does: This paper proposes the EFACTSUM method, which generates multiple candidate summaries and performs dual sorting based on factualness and summary quality. During training, contrastive learning is used to enhance the factualness of abstractive summaries while maintaining or even improving summary quality.
🎯 What it does: Propose two simple techniques to enhance the generalization ability of text-to-SQL semantic parsing based on pre-trained language models: ① Preprocess input tokens to maintain semantic boundaries in subword tokenization; ② Insert special token markers in inputs and outputs to indicate semantic component boundaries aligned with natural language.
Improving the Robustness of Summarization Systems with Dual Augmentation
Xiuying Chen (KAUST), Xiangliang Zhang (University of Notre Dame)
CodeGenerationAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: This study first evaluates the robustness of summary models against word-level perturbations and identifies their vulnerability through adversarial attacks; subsequently, a dual data augmentation strategy is proposed, significantly enhancing the model's stability on noisy and adversarial samples.
Incorporating Attribution Importance for Improving Faithfulness Metrics
Zhixue Zhao (University of Sheffield), Nikolaos Aletras (University of Sheffield)
CodeClassificationExplainability and InterpretabilityTransformerText
🎯 What it does: This paper proposes using a 'soft erase' technique to randomly mask input word vectors in proportion to their importance, thereby improving traditional sparse hard erase (comprehensiveness, sufficiency) evaluation methods when assessing the reliability of feature attribution (FA) methods.
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization
Dongqi Liu (Saarland University), Vera Demberg (Saarland University)
CodeGenerationTransformerLarge Language ModelText
🎯 What it does: Propose RSTformer, which improves long text summarization by leveraging relation types and uncertainty information from Rhetorical Structure Theory (RST)
🎯 What it does: Constructed a fine-grained multidimensional quality assessment (MQM) annotated dataset with 7,000 items to evaluate the quality of machine translation from English to five Indian languages (Hindi, Marathi, Tamil, Malayalam, and Gujarati).
Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling
Shengqiong Wu (National University of Singapore), Tat-Seng Chua (National University of Singapore)
CodeClassificationRepresentation LearningGraph Neural NetworkTransformerVision Language ModelContrastive LearningMultimodality
🎯 What it does: This study addresses multi-modal relation extraction (MRE) by proposing a framework that simultaneously achieves internal information shielding and external information utilization. The framework first represents images and text with visual scene graphs (VSG) and text scene graphs (TSG), respectively, then fuses them into a cross-modal graph (CMG). Subsequently, the CMG is subjected to structure fine-grained screening via graph information bottleneck (GIB) to remove irrelevant noise. Finally, a latent multi-modal topic model (LAMO) is introduced to provide external semantic context for relation extraction, ultimately completing relation prediction.
🎯 What it does: Propose a parameter-efficient multi-layer implicit discourse relation recognition framework (PEMI), achieving multi-layer relation classification with a minimal number of trainable parameters.
🎯 What it does: Propose a training framework called INK, which cyclically injects k-NN knowledge during training using a small number of adapter parameters, gradually smoothing the representation space of neural machine translation models, and removing dependence on large-scale retrieval libraries during inference.
Is Anisotropy Truly Harmful? A Case Study on Text Clustering
Mira Ait-Saada (Université Paris Cité), Mohamed Nadif (Université Paris Cité)
CodeRepresentation LearningTransformerText
🎯 What it does: This paper post-processes the text representations of pre-trained models such as BERT and RoBERTa, and evaluates their representational capacity through document clustering tasks, exploring the impact of spatial anisotropy on clustering performance.
🎯 What it does: Propose and evaluate a distance-based zero-shot OOD detection method without any fine-tuning of pre-trained language models, and compare it with various fine-tuning objectives (CE, TAPT, SupCon) and output layer baseline methods (MSP, energy).
Joint Document-Level Event Extraction via Token-Token Bidirectional Event Completed Graph
Qizhi Wan (Stony Brook University), Rong Hu (Jiangxi University of Finance and Economics)
CodeRecurrent Neural NetworkGraph Neural NetworkTransformerTextFinance Related
🎯 What it does: Proposes a joint document-level event extraction method based on a Token-Token bidirectional event completion graph, and implements the corresponding edge prediction model EDEE.