These 435 EMNLP 2024 papers come with a code repository. Each shows an AI one-line summary below — get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every EMNLP 2024 paper, free trial on arXivSub.
“A good pun is its own reword”: Can Large Language Models Understand Puns?
Zhijun Xu (Fudan University), Deqing Yang (Fudan University)
CodeRecognitionGenerationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Systematically evaluate the capabilities of large language models (LLMs) in understanding puns, covering three tasks: pun identification, explanation, and generation.
“Flex Tape Can’t Fix That”: Bias and Misinformation in Edited Language Models
Karina Halevy (École polytechnique fédérale de Lausanne), Antoine Bosselut (École polytechnique fédérale de Lausanne)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Study the impact of model editing methods on gender, racial, and geographic biases in large language models, and propose a new benchmark dataset SEESAW-CF for evaluation
“We Demand Justice!”: Towards Social Context Grounding of Political Texts
Rajkumar Pujari (Purdue University), Dan Goldwasser (Purdue University)
CodeClassificationGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes and implements two tasks related to social context semantic attribution and sentiment identification for political texts, and constructs corresponding annotated datasets.
A Fast and Sound Tagging Method for Discontinuous Named-Entity Recognition
Caio Corro (INSA Rennes)
CodeRecognitionComputational EfficiencyTransformerTextBiomedical Data
🎯 What it does: Propose a novel tagging scheme based on a two-layer structure, achieving efficient and unambiguous discrete named entity recognition through weighted finite state automata (WFSA).
🎯 What it does: Propose the STAR method, which uses bidirectional KL similarity in log space to guide the distribution of text samples in Euclidean space, enabling fine-grained category discovery without fine-grained annotations.
A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners
Bowen Jiang (University of Pennsylvania), Dan Roth (Argonne National Laboratory)
CodeData SynthesisExplainability and InterpretabilityTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Investigated the 'token bias' of large language models (LLMs) in logical reasoning tasks, i.e., whether models truly reason or merely answer based on vocabulary preferences.
A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models
Houquan Zhou (Soochow University), Min Zhang (Soochow University)
CodeTransformerLarge Language ModelText
🎯 What it does: This paper proposes a Chinese spelling correction method based on large language models (LLM) that requires neither training nor prompting.
A SMART Mnemonic Sounds like “Glue Tonic”: Mixing LLMs with Student Feedback to Make Mnemonic Learning Stick
Nishant Balepur (University of Maryland), Jordan Lee Boyd-Graber
CodeOptimizationComputational EfficiencyData-Centric LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Designed and implemented SMART, a keyword mnemonic generator aligned with student feedback, which fine-tunes LLaMA-2 70B and collects three types of student preferences (comparisons, Likert scales, and learning effectiveness) in a Flashcard application. Finally, a Bayesian model synthesizes effectiveness signals and aligns the model using Direct Preference Optimization (DPO).
🎯 What it does: Systematically reviews and critically evaluates the entire process of evaluating large language models (LLMs), identifying and summarizing the main challenges and limitations in terms of reproducibility, reliability, and robustness during the evaluation process.
A Usage-centric Take on Intent Understanding in E-Commerce
Wendi Zhou (University of Edinburgh), Jeff Z. Pan (University of Edinburgh)
CodeRecommendation SystemTransformerLarge Language ModelTextGraphBenchmark
🎯 What it does: Propose a use-centered intent understanding paradigm and construct a product recovery benchmark evaluation framework, analyzing and quantifying the two major weaknesses of the existing intent knowledge graph FolkScope: attribute ambiguity and category rigidity;
ActPlan-1K: Benchmarking the Procedural Planning Ability of Visual Language Models in Household Activities
Ying Su (Hong Kong University of Science and Technology), Yangqiu Song (University of California San Diego)
CodeTransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmark
🎯 What it does: Introduce the ActPlan-1K benchmark to evaluate the capabilities of vision-language models in multimodal and counterfactual home activity planning;
Adaptable Moral Stances of Large Language Models on Sexist Content: Implications for Society and Gender Discourse
Rongchen Guo (University of Ottawa), Svetlana Kiritchenko (National Research Council Canada)
CodeExplainability and InterpretabilityLarge Language ModelPrompt EngineeringText
🎯 What it does: Explore the moral reasoning capabilities of large language models (LLMs) in explaining implicit gender discriminatory content, both critiquing and defending such content
🎯 What it does: Propose the ADOT (Adaptation-of-Thought) method, which automatically evaluates problem difficulty and dynamically constructs demonstration sets and retrieval strategies based on difficulty to enhance the performance of large language models on reasoning tasks.
Adaptive Axes: A Pipeline for In-domain Social Stereotype Analysis
Qingcheng Zeng (Northwestern University), Rob Voigt
CodeTransformerLarge Language ModelText
🎯 What it does: Propose the ADAPTIVE AXES pipeline, which encodes the context of masked target entities using text embedding models and captures domain-specific social stereotypes through adaptive semantic axes.
Adaptive Immune-based Sound-Shape Code Substitution for Adversarial Chinese Text Attacks
Ao Wang (China University of Petroleum (East China)), Weifeng Liu (China University of Petroleum (East China))
CodeAdversarial AttackText
🎯 What it does: Propose a Chinese text adversarial attack framework that generates natural replacements using sound-shape codes and determines the replacement order through an adaptive immune algorithm.
ADELIE: Aligning Large Language Models on Information Extraction
Yunjia Qi (Tsinghua University), Juanzi Li (Tsinghua University)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: Align and fine-tune large language models for information extraction (IE) tasks, constructing high-quality instruction tuning data IEInstruct and feedback data IEFeedback. ADELIESFT and ADELIEDPO are trained via supervised fine-tuning (SFT) and direct preference optimization (DPO), respectively, and evaluated on zero-shot and few-shot settings across multiple IE tasks, including closed IE, open IE, and on-demand IE.
Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models
Hongfu Liu (National University of Singapore), Michael Shieh (National University of Singapore)
CodeAdversarial AttackTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed the DeGCG framework, which decomposes adversarial suffix search into a preceding behavior-agnostic first-term search (FTS) and a subsequent behavior-related content search (CAS), and further introduced the alternating self-transfer i-DeGCG algorithm; by optimizing the first term to provide high-quality initialization, it significantly enhances the efficiency of suffix attacks on aligned LLMs;
Advancing Event Causality Identification via Heuristic Semantic Dependency Inquiry Network
Haoran Li (Sichuan University), Li Huang (Southwestern University of Finance and Economics)
CodeClassificationComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose the SemDI model, reformulating the event causality identification task as a semantic dependency inquiry problem, generating fill-in words via a fill-in-the-blank Cloze analyzer, and verifying causality between events through cross-attention mechanisms.
Advancing Test-Time Adaptation in Wild Acoustic Test Settings
Hongfu Liu (National University of Singapore), Ye Wang (National University of Singapore)
CodeRecognitionDomain AdaptationTransformerAudio
🎯 What it does: This paper proposes a test-time adaptation (TTA) framework tailored for wild acoustic environments, enabling online model updates for refined acoustic base models (e.g., Wav2vec2, HuBERT, WavLM) under various domain shift scenarios such as noise, accents, and singing.
AgentReview: Exploring Peer Review Dynamics with LLM Agents
Yiqiao Jin (Georgia Institute of Technology), Jindong Wang (University of California Santa Barbara)
CodeTransformerLarge Language ModelAgentic AIPrompt EngineeringText
🎯 What it does: This study constructs the AGENTREVIEW framework through large language model (LLM) agents, fully simulating the five stages of peer review (initial review, author response, reviewer discussion, meta-review, and final decision), and generating over 53,800 review documents. It systematically investigates the impact of multiple factors (reviewer expertise, commitment, intent; area chair style; author anonymity, etc.) on review outcomes.
Aligning Language Models to Explicitly Handle Ambiguity
Hyuhng Joon Kim (Seoul National University), Taeuk Kim (Hanyang University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a four-stage alignment process named APA (Alignment with Perceived Ambiguity), enabling large language models (LLMs) to explicitly identify and handle ambiguous queries by generating clarification requests;
Alignment-Enhanced Decoding: Defending Jailbreaks via Token-Level Adaptive Refining of Probability Distributions
Quan Liu (Beijing University of Posts and Telecommunications), Sen Su (Beijing University of Posts and Telecommunications)
CodeAdversarial AttackLarge Language ModelText
🎯 What it does: Proposed and implemented Alignment-Enhanced Decoding (AED) for large language models to defend against jailbreak attacks, utilizing competitive index and self-assessment to dynamically adjust token probability distributions.
CodeComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: In large language models, based on the Heavy-Tail Self-Regularization theory for layer quality assessment, AlphaLoRA is designed to automatically assign the number of LoRA-MoE experts per layer, reducing redundancy and improving performance.
Altogether: Image Captioning via Re-aligning Alt-text
Hu Xu (Meta FAIR), Christoph Feichtenhofer (Meta FAIR)
CodeGenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: By performing multi-round manual re-alignment on existing alt-text, generating more refined and image-content-aligned descriptions, and training a lightweight captioner to automatically complete this re-alignment process on large-scale images.
An Audit on the Perspectives and Challenges of Hallucinations in NLP
Pranav Narayanan Venkit (Pennsylvania State University), Shomir Wilson (Pennsylvania State University)
CodeTextReview/Survey Paper
🎯 What it does: This paper conducts a systematic audit of 103 NLP papers on hallucinations and surveys 171 NLP and AI professionals to explore the concept, definition, evaluation metrics, and differences in community cognition and practice regarding hallucinations.
🎯 What it does: Propose DiffusionCLS, which enhances sentiment classification data using a diffusion language model, focusing on reconstructing label-related vocabulary to improve low-resource sentiment classification performance.
CodeKnowledge DistillationTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: This paper studies the reasoning distillation method for small multilingual models and proposes a novel d-CoT-nR distillation scheme that utilizes positive and negative reasoning paths to enhance reasoning performance.
An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance
Simran Khanuja (Carnegie Mellon University), Graham Neubig (Carnegie Mellon University)
CodeImage TranslationTransformerLarge Language ModelVision Language ModelDiffusion modelImageTextRetrieval-Augmented Generation
🎯 What it does: Propose and evaluate the novel task of 'image cross-cultural recreation,' constructing three end-to-end, caption+LLM-based, retrieval-style generation pipelines and conducting experiments on a self-made evaluation dataset.
An Inversion Attack Against Obfuscated Embedding Matrix in Language Model Inference
Yu Lin (Bytedance), Bing Duan (Bytedance)
CodeSafty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes an element difference-based nearest neighbor attack (EDNN) targeting the embedding matrix encryption scheme in language model inference, and proves that the existing glide-reflection encryption method is ineffective in protecting user privacy, allowing the complete recovery of the original input text.
ApiQ: Finetuning of 2-Bit Quantized Large Language Model
Baohao Liao (University of Amsterdam), Christof Monz (University of Amsterdam)
CodeComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes ApiQ, an efficient fine-tuning method for low-bit quantized LLMs that achieves joint quantization of model weights and LoRA component initialization.
AppBench: Planning of Multiple APIs from Various APPs for Complex User Instruction
Hongru Wang (Chinese University of Hong Kong), Kam-Fai Wong (Chinese University of Hong Kong)
CodeTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: This paper constructs the AppBench benchmark to evaluate large language models (LLMs) in planning and executing multiple API calls from different apps to satisfy complex user instructions, covering graphical execution order and permission restrictions.
APPLS: Evaluating Evaluation Metrics for Plain Language Summarization
Yue Guo (University of Illinois Urbana-Champaign), Lucy Lu Wang (University of Washington)
CodeGenerationTransformerLarge Language ModelTextBiomedical DataBenchmark
🎯 What it does: Constructed the APPLS meta-evaluation test platform, systematically perturbing four evaluation dimensions (information, simplification, coherence, faithfulness) in the plain language summary (PLS) task, and used this platform to evaluate 14 existing evaluation metrics;
🎯 What it does: This paper systematically evaluates the effectiveness of three intrinsic debiasing techniques (Hard-Debiasing, INLP, LEACE) in neural machine translation (NMT), exploring the impact of debiased embedding layers, tokenization strategies, and differences in target languages.
Are Data Augmentation Methods in Named Entity Recognition Applicable for Uncertainty Estimation?
Wataru Hashimoto (Nara Institute of Science and Technology), Taro Watanabe (Nara Institute of Science and Technology)
CodeRecognitionExplainability and InterpretabilityTransformerText
🎯 What it does: This paper investigates the impact of data augmentation on confidence calibration and uncertainty estimation in named entity recognition (NER), systematically evaluating the effectiveness of various augmentation methods in cross-domain and cross-lingual scenarios.
Are Large Language Models Capable of Generating Human-Level Narratives?
Yufei Tian (University of California Los Angeles), Nanyun Peng
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Examined the capabilities of large language models in narrative generation and understanding, and proposed a quantitative analysis framework based on story arcs, plot points, and emotional dimensions.
Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions
Qian Ruan (Technical University of Darmstadt), Iryna Gurevych (Technical University of Darmstadt)
CodeClassificationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Built and evaluated a large language model (LLM) fine-tuning framework tailored for classification tasks, with a focus on edit intent classification (EIC) in scientific document revisions, and used this framework to automatically annotate and generate a significantly larger Re3-Sci2.0 dataset.
Are Large Language Models In-Context Personalized Summarizers? Get an iCOPERNICUS Test Done!
Divya Patel (KDMLab, Dhirubhai Ambani Institute of Information and Communication Technology), Tanmoy Chakraborty (Indian Institute of Technology Delhi)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Designed and evaluated the iCOPERNICUS framework to examine the In-Context Personalization Learning (ICPL) capability of large language models (LLMs) in context-aware personalized summarization tasks.
ARES: Alternating Reinforcement Learning and Supervised Fine-Tuning for Enhanced Multi-Modal Chain-of-Thought Reasoning Through Diverse AI Feedback
Ju-Seung Byun (Ohio State University), Andrew Perrault (Ohio State University)
CodeReinforcement Learning from Human FeedbackLarge Language ModelSupervised Fine-TuningReinforcement LearningMultimodalityChain-of-Thought
🎯 What it does: A two-stage algorithm alternates between reinforcement learning (RL) and supervised fine-tuning (SFT), leveraging sentence-level scoring and correction feedback from an AI teacher to enhance the quality of multi-modal chain-of-thought reasoning.
ARM: An Alignment-and-Replacement Module for Chinese Spelling Check Based on LLMs
Changchun Liu, Enhong Chen (China University of Mining)
CodeTransformerLarge Language ModelText
🎯 What it does: Designed and implemented an ARM module that leverages LLM for alignment and replacement, enhancing the detection and correction performance of the Chinese Spelling Correction (CSC) system.
🎯 What it does: Propose an adversarial tuning multi-agent system (ATM), which enhances the robustness and generation quality of retrieval-augmented generation models when facing noisy retrieval results through alternating iterative training between an attacker and a generator.
Atomic Inference for NLI with Generated Facts as Atoms
Joe Stacey (Imperial College London), Marek Rei (Imperial College London)
CodeClassificationExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper proposes an atomic reasoning framework that utilizes facts generated by LLMs as atomic units for natural language inference. The model is trained to make entailment/contradiction predictions at the fact level without using fact labels, and instance-level predictions are obtained through deterministic rules.
CodeGenerationLarge Language ModelContrastive LearningText
🎯 What it does: Propose the Atomic Self-Consistency (ASC) method, which generates more complete and higher recall long-text answers by clustering, filtering, and merging atomic facts from multiple generated texts.
Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters
Zhiyu Guo (Nara Institute of Science and Technology), Taro Watanabe (Nara Institute of Science and Technology)
CodeComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose a Token Pruning method that simultaneously utilizes attention scores and value vector norms to evaluate token importance in LLM KV cache compression
Automatic sentence segmentation of clinical record narratives in real-world data
Dongfang Xu (Cedars Sinai Medical Center), Graciela Gonzalez Hernandez (Cedars Sinai Medical Center)
CodeSegmentationTransformerLarge Language ModelTextBiomedical DataElectronic Health RecordsBenchmark
🎯 What it does: Propose a sentence segmentation method based on BERT sequence labeling combined with a dynamic sliding window, and manually annotate 90 clinical notes from MIMIC-III to create the first clinical text sentence segmentation dataset.
AutoPersuade: A Framework for Evaluating and Explaining Persuasive Arguments
Till Raphael Saenger (Princeton University), Brandon M. Stewart (Princeton University)
CodeExplainability and InterpretabilityLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes the AutoPersuade three-step workflow: ① Collect diverse persuasive arguments and audience feedback; ② Build a supervised semi-non-negative matrix factorization (SUN) topic model to extract latent features that explain persuasiveness; ③ Use the model to predict the persuasiveness of new arguments and estimate the causal effects of these features; and validate the process in a case study on vegetarianism.
CodeRepresentation LearningTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: Propose two pixel-based autoregressive pre-training models, PixelGPT and DualGPT, which are pre-trained on visual-text images and text modalities respectively, and explore the effects of joint pre-training across both modalities.
Jonathan Hus (George Mason University), Antonios Anastasopoulos (George Mason University)
CodeTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Improve machine translation for 16 low-resource languages by incorporating a dictionary, grammar book, and a small number of parallel sentences into the prompt using GPT-4-turbo.
🎯 What it does: Propose the RESUME metric to evaluate the quality of candidate translations relative to reference translations, and combine it with existing absolute metrics (e.g., BLEU, COMET) to address the reference bias problem;
BiasWipe: Mitigating Unintended Bias in Text Classifiers through Model Interpretability
Mamta (Indian Institute of Technology Patna), Asif Ekbal (Indian Institute of Technology Jodhpur)
CodeClassificationExplainability and InterpretabilityTransformerText
🎯 What it does: Identify and prune neuron weights by leveraging model explainability (Shapley values) to eliminate unintended bias in text classification models.
Bio-RFX: Refining Biomedical Extraction via Advanced Relation Classification and Structural Constraints
Minjia Wang (Tsinghua University), Jianyong Wang (Tsinghua University)
CodeClassificationRecognitionTransformerPrompt EngineeringBiomedical Data
🎯 What it does: Propose Bio-RFX, a medical text entity and relation extraction framework that first identifies the relationship types present in the sentence and then performs entity extraction through relationship-specific question-answering.
Birdie: Advancing State Space Language Modeling with Dynamic Mixtures of Training Objectives
Sam Blouir (George Mason University), Amarda Shehu (George Mason University)
CodeRetrievalLarge Language ModelReinforcement LearningText
🎯 What it does: Propose the Birdie training method, combining bidirectional forward processing with dynamic mixed pre-training objectives to enhance SSM's performance on retrieval tasks.
BLSP-Emo: Towards Empathetic Large Speech-Language Models
Chen Wang (University of Chinese Academy of Sciences), Jiajun Zhang (University of Chinese Academy of Sciences)
CodeRecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningTextAudio
🎯 What it does: This study proposes BLSP-Emo, an end-to-end emotional speech-language large model capable of understanding semantic and emotional cues in speech and generating empathetic text responses;
Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models?
Siyu Yuan (School of Data Science Fudan University), Deqing Yang (School of Data Science Fudan University)
CodeKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Propose the SCUA task, leveraging teacher large language models to generate analogies to assist student large language models in understanding scientific concepts and answering related multiple-choice questions.
Bootstrapped Policy Learning for Task-oriented Dialogue through Goal Shaping
Yangyang Zhao (Changsha University of Science and Technology), Shihan Wang (Utrecht University)
CodeReinforcement LearningText
🎯 What it does: Propose Bootstrapped Policy Learning (BPL), which dynamically generates sub-goal curricula through goal shaping to address the sparse reward problem in dialogue.
BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training
Pavel Chizhov (Technical University of Applied Sciences Würzburg-Schweinfurt), Ivan P. Yamshchikov (Technical University of Applied Sciences Würzburg-Schweinfurt)
🎯 What it does: Propose PickyBPE, a tokenizer method that dynamically removes intermediate junk tokens during BPE training and achieves efficient vocabulary refinement with a fixed vocabulary size.
Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models
Terra Blevins (University of Washington), Luke Zettlemoyer (University of Washington)
CodeComputational EfficiencyRepresentation LearningTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: This paper proposes a cross-lingual expert language model (X-ELM), which alleviates the 'curse of multilingualism' in multilingual models by dividing large multilingual corpora into subsets and training specialized expert models for each subset.
Bridging Local Details and Global Context in Text-Attributed Graphs
Yaoke Wang (Zhejiang University), Siliang Tang (Zhejiang University)
CodeClassificationGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningTextGraph
🎯 What it does: Design and implement the GraphBridge framework, integrating local text encoding with global graph structure aggregation, and propose a graph-aware token pruning module to enhance efficiency and scalability.
C-LLM: Learn to Check Chinese Spelling Errors Character by Character
Kunting Li (Tsinghua University), Jie Zhou (Tencent Inc)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose C-LLM, which utilizes character-level tokenization to enable large language models to check and correct Chinese spelling errors character by character.
C3PA: An Open Dataset of Expert-Annotated and Regulation-Aware Privacy Policies to Enable Scalable Regulatory Compliance Audits
Maaz Bin Musa (University of Iowa), Rishab Nithyanand (University of Iowa)
CodeSafty and PrivacyData-Centric LearningTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: This study created the first expert-annotated privacy policy dataset based on CCPA called C3PA, and used it to train a model for compliance checks.
Can Automatic Metrics Assess High-Quality Translations?
Sweta Agrawal (Instituto de Telecomunicações), Andre Martins
CodeClassificationTextBenchmark
🎯 What it does: Systematically evaluate the ability of existing machine translation evaluation metrics to identify high-quality translations with no errors (zero MQM), focusing on their performance in distinguishing subtle quality differences under the same source sentence.
Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?
Zhe Yang (Peking University), Zhifang Sui (Alibaba Group)
CodeExplainability and InterpretabilityTransformerLarge Language ModelTextBenchmark
🎯 What it does: Constructed the ConsisEval benchmark to study the hard-easy consistency issue of LLMs, i.e., whether LLMs can solve easier problems when solving harder ones, and proposed two metrics: consistency score (CS) and relative consistency score (RCS).
🎯 What it does: This work constructs a large multi-annotated dataset named Casablanca, containing 48 hours of speech with eight Arabic dialects, gender, and code-switching labels, and evaluates multilingual ASR models on it.
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification
Junhui He (Wuhan University), Qingan Li (Wuhan University)
CodeComputational EfficiencyTransformerText
🎯 What it does: To address the inference efficiency of large language models on edge devices, the CHESS method is proposed to achieve activation sparsification by implementing channel-level threshold pruning and selective sparsification on the feed-forward network (FFN) and attention modules.
🎯 What it does: Constructed an expert-annotated ClimRetrieve dataset and evaluated the performance of different embedding retrieval strategies in corporate climate disclosure question-answering tasks.
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation
Yuan Ge (Northeastern University), JingBo Zhu
CodeData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: During the supervised fine-tuning phase, this paper addresses the issues of instruction data quality and diversity by proposing a method for selecting instructions based on quality assessment and clustering. It first evaluates the quality of instruction pairs using an expert-aligned scoring model, then retains diversity through clustering, filtering a high-quality subset from a large-scale instruction set for fine-tuning.
CoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing
Xinyi He (Xi'an Jiaotong University), Dongmei Zhang (Microsoft Research)
CodeAI Code AssistantTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose the CoCoST framework, integrating online search with query planning, automatically generated test cases, and input/output serialization to enhance the quality of large language models (LLMs) in complex code generation.
CodeJudge: Evaluating Code Generation with Large Language Models
Weixi Tong (Huazhong University of Science and Technology), Tianyi Zhang (Purdue University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Proposed and implemented a framework called CODEJUDGE that utilizes large language models (LLMs) to evaluate the semantic correctness of code generation results without test cases;
CoEvol: Constructing Better Responses for Instruction Finetuning through Multi-Agent Cooperation
Renhao Li (University of Macau), Min Yang (Shenzhen Institute of Advanced Technology)
CodeData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningAgentic AIPrompt EngineeringTextBenchmark
🎯 What it does: Proposes the CoEvol framework, which employs LLM-based multi-agent systems (debaters, advisors, editors, judges) through a debate-suggestion-edit-judgment cycle to iteratively improve responses in instruction fine-tuning data, thereby generating higher-quality training samples.
CoGen: Learning from Feedback with Coupled Comprehension and Generation
Mustafa Omer Gul (Cornell University), Yoav Artzi (Cornell University)
CodeGenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: Designed and implemented a system (COGEN) capable of simultaneously performing language understanding and generation, and continuously learning through human interaction, deployed in a two-player dialogue game.
Collaborative Performance Prediction for Large Language Models
Qiyuan Zhang (City University of Hong Kong), Chen Ma (City University of Hong Kong)
CodeOptimizationExplainability and InterpretabilityData-Centric LearningTabularBenchmark
🎯 What it does: Propose a collaborative performance prediction framework (CPP) that predicts performance on various downstream tasks by leveraging historical LLM performance and design factors.
CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions
Jun Rao (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Proposes CommonIT, an instruction tuning strategy that partitions data based on commonalities (task, embedding, length) and samples within a single group, enhancing LLM's instruction following ability.
Commonsense Knowledge Editing Based on Free-Text in LLMs
Xiusheng Huang (Key Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation Chinese Academy of Sciences), Kang Liu (Key Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation Chinese Academy of Sciences)
CodeTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes an editing method for free-text form common sense knowledge in large language models and constructs the corresponding benchmark dataset CKEBench. Through the knowledge localization experiment (KLFT), it reveals that common sense knowledge is distributed in MLP and attention layers and is dispersed. Subsequently, a Dynamics-aware Editing Method (DEM) is designed, which includes a dynamic perception module and a knowledge editing module, achieving precise editing of common sense knowledge. Finally, experiments are conducted on GPT-J(6B) and LLaMA-2(7B), demonstrating that DEM significantly outperforms existing editing methods in metrics such as Score and Commonsense.
Communicating with Speakers and Listeners of Different Pragmatic Levels
Kata Naszadi, Christof Monz (University of Amsterdam)
CodeVision Language ModelMultimodalityChain-of-Thought
🎯 What it does: Simulate language learning and interaction, investigating how varying reasoning depths between speaker and listener impact communication success
CompAct: Compressing Retrieved Documents Actively for Question Answering
Chanwoong Yoon (Korea University), Jaewoo Kang (Korea University)
CodeRetrievalCompressionTransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: Proposed the COMPACT framework, which actively compresses retrieved multiple documents and iteratively generates a concise answer context by integrating the question context during compression until the judgment of 'sufficient to answer the question' is met;
🎯 What it does: Propose a retrieval and re-ranking framework named Comparing Multiple Candidates (CMC), which improves candidate context representations by parallelly comparing multiple candidate vectors within the self-attention layer;
Qiwei Peng (University of Copenhagen), Anders Søgaard (University of Copenhagen)
CodeRetrievalRepresentation LearningTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: By treating parallel concepts in multilingual WordNet as dictionaries, conducting linear alignment experiments on multilingual large language models to evaluate the quality of their concept space alignment.
Concept-skill Transferability-based Data Selection for Large Vision-Language Models
Jaewoo Lee (KAIST), Sung Ju Hwang (KAIST)
CodeData-Centric LearningVision Language ModelMultimodality
🎯 What it does: Propose the COINCIDE method, which clusters visual instruction tuning data using internal activations of a small vision-language model and selects training samples based on the transitivity and density of the clusters;
Consistent Autoformalization for Constructing Mathematical Libraries
Lan Zhang (University of Manchester), Andre Freitas (University of Manchester)
CodeAI Code AssistantTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed an automated formalization framework based on large language models, integrating three mechanisms: retrieval-augmented generation, denoising, and syntax error feedback to build consistent and scalable mathematical libraries.
Continual Test-time Adaptation for End-to-end Speech Recognition on Noisy Speech
Guan-Ting Lin (National Taiwan University), Hung-yi Lee (National Taiwan University)
CodeRecognitionDomain AdaptationTransformerAudio
🎯 What it does: Propose the Fast-slow TTA framework and the DSUTA method to address performance degradation of Continual Test-time Adaptation (CTTA) on multi-domain noisy speech;
🎯 What it does: The study addresses the challenges of entity coreference and disambiguation in historical texts, proposing a bi-encoder model based on contrastive learning, along with providing large-scale training data and historical benchmark evaluations.
Contribution of Linguistic Typology to Universal Dependency Parsing: An Empirical Investigation
Ali Basirat (University of Copenhagen), Navid Baradaran Hemmati (Certified Translation Agency No. 1141)
CodeData-Centric LearningText
🎯 What it does: Applied Croft's semantic transformation rules to the Universal Dependencies treebank to construct Typologically-informed Universal Dependencies (TUD), and evaluated its impact on dependency parsing accuracy.
Yiju Guo (Renmin University of China), Maosong Sun (Tsinghua University)
CodeOptimizationReinforcement Learning from Human FeedbackLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: Propose a controllable preference optimization (CPO) method, using preference tokens to control the output of large models in multiple objectives such as helpfulness, honesty, and harmlessness, achieving controllable multi-objective alignment.
CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation
Tong Chen (University of Washington), Pang Wei Koh (University of Washington)
CodeGenerationTransformerLarge Language ModelTextBenchmark
🎯 What it does: This study proposes the COPYBENCH benchmark for automatically evaluating literal and non-literal copying of language models on copyrighted texts, while simultaneously measuring practical metrics such as factual recall and fluency.
Yike Wu (Southeast University), Jeff Z. Pan (University of Edinburgh)
CodeTransformerLarge Language ModelGraphBenchmarkChain-of-Thought
🎯 What it does: Proposed the CoTKR method, which enhances knowledge rewriting with chain-of-thought (CoT) reasoning to generate more useful natural language knowledge representations, thereby improving the performance of knowledge graph question answering (KGQA).
🎯 What it does: This study constructs a dataset (CD-ADD) containing over 300 hours of cross-domain zero-shot TTS synthesized speech, and evaluates and enhances the deepfake audio detection performance of Wav2Vec2 and Whisper under various attacks (noise, echo, compression, etc.) through attack augmentation training and few-shot fine-tuning methods.
CodeRecognitionDomain AdaptationTransformerLarge Language ModelText
🎯 What it does: Generate task-oriented knowledge (GTOK) automatically using large language models (LLMs) and employ it for masked span language modeling pre-training to construct a cross-domain named entity recognition (CDNER) framework called TOPT.
CURE: Context- and Uncertainty-Aware Mental Disorder Detection
Migyeong Kang (Sungkyunkwan University), Jinyoung Han (Sungkyunkwan University)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed the CURE framework, which detects mental disorders by leveraging symptoms, contextual information, and uncertainty fusion, and constructed the KoMOS Korean mental health dataset.
CUTE: Measuring LLMs’ Understanding of Their Tokens
Lukas Edman (LMU Munich), Alexander Fraser (TU Munich)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Designed the CUTE benchmark to evaluate large language models' morphological understanding and character-level manipulation capabilities regarding subword tokenization.
DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models
Yiming Huang (Key Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation Chinese Academy of Sciences), Kang Liu (Key Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation Chinese Academy of Sciences)
CodeAI Code AssistantTransformerLarge Language ModelAgentic AITextTabularBenchmark
🎯 What it does: Proposed the DA-Code benchmark to evaluate the capabilities of large language models in agent-based data science code generation tasks; simultaneously constructed the DA-Agent framework as an execution and interaction code agent;
Decoding the Echoes of Vision from fMRI: Memory Disentangling for Past Semantic Information
Runze Xia (Nanjing University of Aeronautics and Astronautics), Piji Li (Nanjing University of Aeronautics and Astronautics)
CodeRepresentation LearningVision Language ModelContrastive LearningBiomedical DataMagnetic Resonance Imaging
🎯 What it does: Under continuous visual stimuli, this study investigates the retention of past visual information in working memory using fMRI data, and proposes the Memory Disentangling task, aiming to extract and decouple current and past semantic information from brain signals at a single time point.
Decoding with Limited Teacher Supervision Requires Understanding When to Trust the Teacher
Hyunjong Ok (Pohang University of Science and Technology), Jaeho Lee (Pohang University of Science and Technology)
CodeComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelImageTextBenchmarkAudio
🎯 What it does: Propose that in scenarios with limited teacher supervision, small-scale LLMs can improve generation quality by using adaptive α mixing and entropy threshold strategies with minimal supervision from large models.
DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models
Ranchi Zhao (Modelbest Inc), Maosong Sun (Tsinghua University)
CodeKnowledge DistillationData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposed and implemented DecorateLM, a complete workflow for sample-level data engineering on large-scale pre-trained corpora through text scoring, hierarchical labeling, and editing;
🎯 What it does: Proposed the DEFT-UCS framework, which employs unsupervised core-set selection to perform data-efficient fine-tuning of pre-trained language models (PLMs) for text editing tasks.