These 356 ACL 2024 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every ACL 2024 paper, free trial on arXivSub.
\inftyBench: Extending Long Context Evaluation Beyond 100K Tokens
Xinrong Zhang (Tsinghua University), Maosong Sun (Tsinghua University)
CodeData SynthesisRetrievalAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
π― What it does: Proposed the βBENCH benchmark, covering multi-domain bilingual long-context tasks with an average of over 100K tokens.
A Causal Approach for Counterfactual Reasoning in Narratives
Feiteng Mu (Hong Kong Polytechnic University), Wenjie Li (Hong Kong Polytechnic University)
CodeGenerationData SynthesisTransformerLarge Language ModelReinforcement LearningAuto EncoderContrastive LearningTextRetrieval-Augmented Generation
π― What it does: Propose a causal framework based on Variational Autoencoder (VAE) to generate counterfactual stories consistent with given scenarios, enhanced by a pre-trained entailment classifier and external event commonsense knowledge.
A Joint Coreference-Aware Approach to Document-Level Target Sentiment Analysis
Hongjie Cai (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
CodeClassificationTransformerLarge Language ModelText
π― What it does: This paper proposes a joint coreference-aware document-level target sentiment analysis framework, which utilizes dual-path RoBERTa multi-task learning to simultaneously perform target extraction, sentiment classification, and hierarchical relationship inference, while enhancing model performance through coreference information.
A Modular Approach for Multimodal Summarization of TV Shows
Louis Mahon (University of Edinburgh), Mirella Lapata (University of Edinburgh)
CodeGenerationTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
π― What it does: Built a modular multi-modal long-form TV program summarization system, decomposing the task into five subtasks: scene detection, scene reordering, visual-to-text conversion, dialogue summarization, and high-level summary fusion.
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Any Translation
Zhengrui Ma (Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences), Min Zhang (Soochow University)
π― What it does: Proposes a non-autoregressive end-to-end simultaneous interpretation framework (NAST-S2x) that can simultaneously handle speech-to-text (Simul-S2T) and speech-to-speech (Simul-S2S) tasks, achieving real-time generation through a block-level encoder + block-level non-autoregressive decoder.
π― What it does: Studied the natural language inference (NLI) task for the Romanian language, created the first RoNLI corpus, and designed a curriculum learning strategy based on data visualization methods.
A Sentiment Consolidation Framework for Meta-Review Generation
Miao Li (University of Melbourne), Eduard Hovy (University of Melbourne)
CodeTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: This study proposes a three-tier sentiment merging framework for automatically generating meta-reviews in academic peer review, and designs new prompting methods and evaluation metrics based on this framework;
A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts
Nafis Irtiza Tripto (Pennsylvania State University), Dongwon Lee (Pennsylvania State University)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper systematically evaluates the authorship attribution problem of large language models (LLM) on text after multi-round rewriting, exploring whether rewriting leads to changes in author identity;
ABEX: Data Augmentation for Low-Resource NLU via Expanding Abstract Descriptions
Sreyan Ghosh (University of Maryland), Dinesh Manocha (University of Maryland)
CodeClassificationRecognitionData SynthesisData-Centric LearningGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose the ABEX method, which first converts documents into concise abstract descriptions and then uses BART to expand these abstractions, generating diverse augmented samples.
CodeSafty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper proposes a defense method called MuScleLoRA against backdoor attacks, which trains language models using frequency-domain multi-scale scaling and low-rank adapters (LoRA) to suppress the learning of low-frequency features from backdoor mappings, enhance the learning priority of clean mappings, and thus train a clean model on poisoned data.
ActionIE: Action Extraction from Scientific Literature with Programming Languages
Xianrui Zhong (University of Illinois Urbana-Champaign), Jiawei Han (University of Illinois Urbana-Champaign)
CodeGenerationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This study proposes the ActionIE method, which utilizes large language models to convert experimental procedures in chemical literature into executable Python code, achieving structured extraction of experimental steps.
Advancing Abductive Reasoning in Knowledge Graphs through Complex Logical Hypothesis Generation
Jiaxin Bai (Hong Kong University of Science and Technology), Yangqiu Song (Hong Kong University of Science and Technology)
CodeGenerationOptimizationExplainability and InterpretabilityTransformerReinforcement LearningGraph
π― What it does: Studied the task of abductive reasoning on knowledge graphs, proposing a method to generate complex logical hypotheses from given observations, and enhancing the interpretability of hypotheses through reinforcement learning.
AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators
Jingwei Ni (ETH ZΓΌrich), Markus Leippold (University of ZΓΌrich)
CodeClassificationData-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
π― What it does: Propose the AFaCTA framework, which leverages large language models (LLMs) to assist in the annotation work for fact claim detection.
AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts
Daniel Braun (University of Twente), Florian Matthes (Technical University of Munich)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Created the AGB-DE corpus, containing 3,764 German consumer contract clauses, manually annotated for clause legality (potentially invalid/valid) and topic labels, along with baseline evaluation.
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Da Yin (UCLA), Bill Yuchen Lin (Allen Institute for AI)
CodeTransformerLarge Language ModelSupervised Fine-TuningAgentic AIPrompt EngineeringTextMultimodality
π― What it does: Developed an open-source, unified, and modular language agent framework called LUMOS, utilizing the LLAMA-2 7B/13B model and trained with unified annotations to support multi-task interaction;
Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization
Wenqi Zhang (Zhejiang University), Weiming Lu (Zhejiang University)
CodeOptimizationTransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringWorld ModelTextSequential
π― What it does: Propose Agent-Pro, a self-evolving agent that leverages large language models to learn and improve behavioral policies in interactive environments through dynamic belief updates, strategy-level reflection, and DFS optimization;
AI βNewsβ Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italian
Giovanni Puccetti (Istituto di Scienza e Tecnologia dell'Informazione A Faedo), Andrea Esuli (Istituto di Scienza e Tecnologia dell'Informazione A Faedo)
CodeGenerationData SynthesisAnomaly DetectionTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Fine-tuning Llama (and its subsequent models) using only about 40,000 Italian news articles can generate news content nearly indistinguishable from human writing, forming so-called 'content farms'.
AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension
Qian Yang (Zhejiang University), Jingren Zhou (Alibaba Group)
CodeLarge Language ModelTextMultimodalityBenchmarkAudio
π― What it does: Propose AIR-Bench, the first generative evaluation benchmark for large-scale audio-lingual models, containing 19 multiple-choice tasks and over 2000 open-ended questions, covering speech, natural sounds, and music, and introducing volume and time-shifted mixing strategies.
An Entropy-based Text Watermarking Detection Method
Yijian Lu (Chinese University of Hong Kong), Irwin King (Chinese University of Hong Kong)
CodeAnomaly DetectionText
π― What it does: This paper proposes a text watermark detection method based on token entropy (EWD), which significantly improves detection accuracy in low-entropy scenarios by assigning different detection weights to tokens with varying entropy.
An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
Ziwei Chai (Zhejiang University), Yang Yang (Zhejiang University)
CodeLarge Language ModelMixture of ExpertsText
π― What it does: Built a unified expert LLM collaboration framework called Expert-Token-Routing, which encodes expert LLMs into the meta LLM's vocabulary through dedicated tokens, enabling seamless collaboration and routing among multiple expert LLMs.
An Information-Theoretic Approach to Analyze NLP Classification Tasks
Luran Wang (University of Cambridge), Vatsal Raina (University of Cambridge)
CodeClassificationTransformerLarge Language ModelText
π― What it does: Propose an information-theoretic framework that quantifies the impact of input elements (semantic and linguistic realization) on the output distribution in text classification tasks using mutual information.
Analysing The Impact of Sequence Composition on Language Model Pre-Training
Yu Zhao (University of Edinburgh), Pasquale Minervini (University of Edinburgh)
CodeExplainability and InterpretabilityComputational EfficiencyRepresentation LearningData-Centric LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: Analyze the impact of pre-training sequence composition on language models, compare different packing and masking strategies, and propose the BM25Chunk retrieval-based packing method.
Analyzing Semantic Change through Lexical Replacements
Francesco Periti, Nina Tahmasebi
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: This paper constructs lexical substitutions with different semantic relationships (synonyms, antonyms, hypernyms, random words) to simulate semantic changes and observe the embedding distance variations of pre-trained language models in different contexts, thereby quantifying the model's tension towards semantic evolution, and proposes an interpretable method for semantic change detection based on this.
π― What it does: This paper proposes the Angle-optimized Embedding (AoE) model, which splits text embeddings into real and imaginary parts, computes angle differences in complex space, and introduces an angle loss to address the gradient disappearance problem caused by the saturation region of cosine similarity.
API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs
Kinjal Basu (IBM Research), Luis Lastras
CodeData SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextSequentialBenchmark
π― What it does: This paper proposes and constructs the API-BLEND dataset, providing multi-domain, real-world API call data for training and evaluating tool-enhanced large language models.
CodeAI Code AssistantLarge Language ModelAgentic AITextTabularBenchmark
π― What it does: Built the AppWorld Engine and AppWorld Benchmark, providing a controllable multi-app simulation environment and 750 complex interactive coding tasks.
π― What it does: This paper analyzes naturally occurring partial diacritics (WildDiacs), constructs a new maximized diacritics dataset called Wild2MaxDiacs, and improves the analysis-disambiguation workflow of CAMeL Tools. By leveraging WildDiacs for re-ranking and contextual post-processing, high-quality re-diacritization across multiple corpora is achieved.
ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models
Hojae Han (Seoul National University), Seung-won Hwang (Seoul National University)
CodeAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Propose the ARCHCODE framework, which leverages context learning of large language models to automatically extract and infer functional (FR) and non-functional requirements (NFR) from natural language requirements, using them as conditions to generate code and corresponding test cases, followed by filtering and sorting the generated code through executing the test cases;
Are LLM-based Evaluators Confusing NLG Quality Criteria?
Xinyu Hu (Wangxuan Institute of Computer Technology, Peking University), Xiaojun Wan (Wangxuan Institute of Computer Technology, Peking University)
CodeGenerationExplainability and InterpretabilityAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Conduct fine-grained perturbation attack testing on the reliability of large language models (LLMs) in natural language generation (NLG) evaluation, revealing their confusion and over-sensitivity in distinguishing different evaluation dimensions (e.g., fluency, consistency, factual accuracy).
Argument Mining in Data Scarce Settings: Cross-lingual Transfer and Few-shot Techniques
Anar Yeginbergen (University of Basque Country), Rodrigo Agerri (University of Basque Country)
CodeDomain AdaptationData-Centric LearningTransformerSupervised Fine-TuningPrompt EngineeringTextBiomedical Data
π― What it does: Address the data scarcity problem in argument mining within medical text, comparing cross-lingual transfer, data transfer, and few-shot learning methods.
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews
Mike DβArcy, Doug Downey (Allen Institute for AI)
CodeClassificationGenerationTransformerLarge Language ModelText
π― What it does: Propose and evaluate two new tasks: comment-edit alignment of scientific papers based on peer reviews, and edit generation based on peer reviews.
ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs
Fengqing Jiang (University of Washington), Radha Poovendran (University of Washington)
CodeSafty and PrivacyAdversarial AttackLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: This study investigates ASCII art-based jailbreak attacks, proposing the ArtPrompt method that exploits LLMs' insufficient recognition of ASCII art to bypass security filters.
CodeTransformerLarge Language ModelTextMultimodality
π― What it does: Propose AUTOACT, an automated QA agent learning framework from scratch without requiring large-scale labeled data or closed-source models.
BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents
Yifei Wang (Zhengzhou University), Shengsheng Qian (Zhengzhou University)
CodeAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningAgentic AIText
π― What it does: Proposed a backdoor attack named BadAgent targeting large language model agents, demonstrating both active and passive triggering methods, which can implant backdoors during the fine-tuning phase and execute hidden operations after deployment.
Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations
Jiaxing Sun (State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University), Conghui He (Shanghai AI Laboratory)
π― What it does: Created and evaluated the Chinese common sense reasoning benchmark CHARM, covering both global and China-specific domains, and systematically assessed the reasoning and memory capabilities of 19 LLMs and 5 prompting strategies.
Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation
Xunjian Yin (Wangxuan Institute of Computer Technology, Peking University), Xiaojun Wan (Wangxuan Institute of Computer Technology, Peking University)
CodeLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: To address knowledge evaluation in large language models, this paper proposes an evaluation framework based on knowledge boundaries and designs the PGDC algorithm to automatically identify optimal prompts.
Xiaochen Gao (University of California San Diego), Jingbo Shang (University of California San Diego)
CodeClassificationGraph Neural NetworkLarge Language ModelPrompt EngineeringTextGraphChain-of-Thought
π― What it does: Propose a fine-grained dependency graph (FLAN Graph) based on patent claims, and represent it using graph neural networks (GNNs) to achieve patent approval prediction; meanwhile, systematically evaluate the impact of LLM scaling and prompting engineering on this task, proving that traditional LLMs cannot significantly improve performance.
Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation
ZdenΔk Kasner (Charles University), Ondrej Dusek (Charles University)
CodeGenerationTransformerLarge Language ModelPrompt EngineeringTextTabularBenchmark
π― What it does: This paper constructs the QUINTD tool on unlabeled real-world structured data and generates the QUINTD-1 dataset using this tool. It investigates the behavior of open-source large language models (LLMs) in data-to-text generation tasks, and evaluates semantic accuracy through human evaluation and reference-free metrics based on GPT-4.
BinaryAlign: Word Alignment as Binary Sequence Labeling
Gaetan Latouche, Benjamin Swanson
CodeClassificationRepresentation LearningTransformerLarge Language ModelText
π― What it does: Propose BinaryAlign, redefining the word alignment task as a binary sequence labeling problem for each word pair, and uniformly handling high- and low-resource language pairs.
CodeReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This study proposes a black-box prompt optimization (BPO) method to automatically rewrite user prompts to enhance the alignment of LLMs with human preferences without modifying model parameters.
Can ChatGPTβs Performance be Improved on Verb Metaphor Detection Tasks? Bootstrapping and Combining Tacit Knowledge
Cheng Yang (Guangxi University), Qingbao Huang (Guangxi University)
CodeRecognitionTransformerLarge Language ModelText
π― What it does: Propose an unsupervised method for detecting verb metaphors by automatically generating verb semantic collocation tables and theme mappings using ChatGPT.
Can Large Language Models Interpret Noun-Noun Compounds? A Linguistically-Motivated Study on Lexicalized and Novel Compounds
Giulia Rambelli (University of Bologna), Marianna Bolognesi (University of Bologna)
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
π― What it does: The study evaluates the ability of large language models (Llama-2, Falcon, Mistral) to explain noun-noun compounds (with paraphrases indicating relational meanings) and investigates whether models can abstractly infer from known compounds to novel compounds.
Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Models
Zhiwei He (Shanghai Jiao Tong University), Rui Wang (Shanghai Jiao Tong University)
CodeSafty and PrivacyTransformerLarge Language ModelText
π― What it does: Studied the consistency of large language model text watermarks after translation into different languages, proposed a cross-language watermark removal attack (CWRA), and improved the SIR algorithm through cross-language semantic clustering to obtain the X-SIR defense method.
CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning
Weiqi Wang (Amazon.com Inc), Yangqiu Song (Hong Kong University of Science and Technology)
CodeKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Propose the CANDLE framework, which iteratively uses large language models for conceptualization and instantiation to expand the commonsense knowledge base
Causal-Guided Active Learning for Debiasing Large Language Models
Zhouhao Sun (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
CodeExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelPrompt EngineeringText
π― What it does: This paper proposes a causal-guided active learning framework (CAL), which utilizes large language models to automatically identify and induce dataset bias instances, and debiases the model through prompt learning methods;
Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages
Shih-Cheng Huang (National Applied Research Laboratories), Hung-yi Lee (National Taiwan University)
CodeTransformerLarge Language ModelText
π― What it does: Achieve instruction following, dialogue capabilities, and value alignment for large language models in new languages by utilizing continual pre-training (CP) and directly adding chat vectors (Chat Vector) to model weights.
ChatDev: Communicative Agents for Software Development
Chen Qian (Tsinghua University), Maosong Sun (Tsinghua University)
CodeAI Code AssistantTransformerLarge Language ModelAgentic AIPrompt EngineeringText
π― What it does: Proposes ChatDev, a multi-agent collaborative software development framework built using large language models (LLMs). The framework breaks down software lifecycle stages such as design, coding, and testing into subtasks through a 'chat chain,' and reduces hallucination issues in code generation via a 'communicative dehallucination' mechanism.
ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences
Yuanhe Tian (University of Washington), Yongdong Zhang (University of Science and Technology of China)
CodeReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningTextBiomedical Data
π― What it does: Proposed and implemented CHIMED-GPT, a large language model specifically designed for Chinese medical text processing, which employs a three-stage training process consisting of full pre-training, supervised fine-tuning, and reinforcement learning with human feedback.
π― What it does: Propose a long-sequence processing framework named SimCAS, which significantly reduces the computational and storage complexity of self-attention by splitting the input into multiple blocks, performing batch alignment of start/end tokens across blocks in the encoding layer, and using reinforcement learning strategies to select the most representative hidden states during decoding.
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition
Lu Ye (Microsoft), Yang Li (Microsoft)
CodeComputational EfficiencyTransformerLarge Language ModelText
π― What it does: Designed and implemented a prefix-aware KV cache (PAKV) and a two-phase partitioning (TPP) self-attention module, significantly improving memory utilization and throughput in LLM inference.
CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models
Tong Zhang (Sichuan University), Tat-Seng Chua (National University of Singapore)
CodeTransformerLarge Language ModelTextBenchmarkChain-of-Thought
π― What it does: Propose the CLAMBER benchmark to evaluate large language models (LLMs) in identifying and clarifying ambiguous queries, covering a new three-dimensional eight-class ambiguity classification system;
π― What it does: This paper constructs a dataset of 95K lines of movie and TV show dialogues/audio, manually annotated with characters' socioeconomic status, race, and geographic variants, and systematically evaluates the performance differences of NLP tasks such as language models, automatic speech recognition (ASR), and grammar correction across different socioeconomic classes on this dataset; differences are quantified using metrics like WER, perplexity, and edit distance.
CodeData-Centric LearningTransformerLarge Language ModelText
π― What it does: Propose NeuScraper, a web text extractor based on shallow neural networks, capable of efficiently and accurately extracting main content from HTML pages.
Co-training for Low Resource Scientific Natural Language Inference
Mobashir Sadat (University of Illinois Chicago), Cornelia Caragea (University of Illinois Chicago)
CodeClassificationData-Centric LearningTransformerLarge Language ModelText
π― What it does: Propose a co-training method based on training dynamic weighting, which learns low-resource scientific NLI tasks by leveraging a small amount of human-annotated data and a large amount of automatically annotated data generated via distant supervision.
CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
Shiyi Zhu (Ant Group), Jianguo Li (Ant Group)
CodeComputational EfficiencyTransformerLarge Language ModelText
π― What it does: Propose a novel attention mechanism called CoCA, seamlessly integrating position encoding with self-attention to address abnormal behaviors during the extension of long context windows.
π― What it does: Proposed the repo-level code generation task and built the CODEAGENT framework, leveraging a toolset to enhance the generation capability of LLMs in real code repositories.
CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Models
Zixin Chen (Beijing University of Posts and Telecommunications), Guang Chen (Beijing University of Posts and Telecommunications)
CodeClassificationObject DetectionTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityChain-of-Thought
π― What it does: Proposed the CofiPara framework, which leverages multimodal language models (LMM) to generate competitive reasoning (rationales). It first undergoes coarse-grained pre-training on the multimodal sarcasm detection (MSD) task, then undergoes fine-grained fine-tuning on the multimodal sarcasm target identification (MSTI) task, ultimately achieving multimodal sarcasm target detection with interpretable reasoning text for both text and images.
CodeGenerationSafty and PrivacySupervised Fine-TuningTextRetrieval-Augmented Generation
π― What it does: Propose the CoGenesis framework, which collaborates between large models in the cloud and small models locally to achieve context-aware instruction generation while protecting user privacy.
CodeGenerationRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextGraph
π― What it does: Constructed COKE (Cognitive Knowledge Graph) and COLM (Cognitive Language Model) to enable machines to perform theoretical mind reasoning.
Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning
Philipp Mondorf (LMU Munich), Barbara Plank (LMU Munich)
CodeTransformerLarge Language ModelTextChain-of-Thought
π― What it does: Systematic comparison and analysis of the reasoning strategies used by humans and large language models (LLMs) in propositional logic reasoning tasks.
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
Francesco Ortu (University of Trieste), Bernhard SchΓΆlkopf (Max Planck Institute for Intelligent Systems)
CodeExplainability and InterpretabilityTransformerLarge Language ModelText
π― What it does: Investigate the mechanism competition of large language models in processing factual and counterfactual information, using logit inspection and attention modification to track the interaction between factual memory and copying mechanisms, and significantly improve fact recall through a minimal number of attention entries.
Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
Abhishek Kumar (Brock University), Ali Emami (Brock University)
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
π― What it does: Investigate the correspondence between internal confidence in large language models and the confidence expressed by the model (Confidence-Probability Alignment) and evaluate its reliability
Consistency Training by Synthetic Question Generation for Conversational Question Answering
Hamed Hematian Hemati (Sharif University of Technology), Hamid Beigy (Sharif University of Technology)
CodeData SynthesisTransformerText
π― What it does: Enhancing the robustness of conversational question answering models against irrelevant historical information by augmenting conversation history with synthetic questions during training and employing consistency training.
π― What it does: This paper proposes a semi-supervised contrastive distillation (SCD) framework that allows incrementally expanding neural machine translation models to adapt to new domains without accessing old domain training data.
Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor
Sangwon Yu (Seoul National University), Sungroh Yoon (Seoul National University)
CodeGenerationScore-based ModelText
π― What it does: Propose a Score-based Progressive Editor (ScoPE), which achieves control over target attributes by editing intermediate text block-by-block during the generation process of a black-box language model, while maintaining text fluency.
π― What it does: Propose an end-to-end ASR model called CopyNE, which directly copies complete entities from a pre-built named entity (NE) dictionary during decoding, avoiding entity incompleteness or errors caused by character-by-character generation.
CQIL: Inference Latency Optimization with Concurrent Computation of Quasi-Independent Layers
Longwei Zou (Tsinghua University), Yangdong Deng (Tsinghua University)
CodeComputational EfficiencyTransformerLarge Language ModelText
π― What it does: By identifying 'quasi-independent layers' with similar inputs in adjacent layers of LLMs, parallel computation of these layers is achieved, significantly reducing inference latency.
Crayon: Customized On-Device LLM via Instant Adapter Blending and Edge-Server Hybrid Inference
Jihwan Bang (Qualcomm AI Research), Simyung Chang (Qualcomm AI Research)
CodeSafty and PrivacyComputational EfficiencyRepresentation LearningTransformerLarge Language ModelTextBenchmark
π― What it does: Propose the Crayon scheme, achieving LLM customization on the device without training through immediate hybrid LoRA pool, combined with device-server hybrid inference to enhance performance
CSCD-NS: a Chinese Spelling Check Dataset for Native Speakers
Yong Hu (Tencent Inc.), Jie Zhou (Tencent Inc.)
CodeData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
π― What it does: Proposed CSCD-NS β the first spelling correction dataset tailored for native Chinese speakers, generated high-quality pseudo data by simulating input method editors (IME), aiming to enhance model performance in real-world scenarios.
π― What it does: Propose Oracle Bone Script Decipher (OBSD) based on conditional diffusion models, utilizing local structure sampling and zero-shot refinement to map ancient characters to modern Chinese characters.
Decoder-only Streaming Transformer for Simultaneous Translation
Shoutao Guo (Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences), Yang Feng (Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences)
CodeGenerationTransformerText
π― What it does: Proposed the first real-time translation model DST (Decoder-only Streaming Transformer) based on a Decoder-only architecture, achieving end-to-end adaptive translation strategies by separating source/target prefix position encodings and designing Streaming Self-Attention (SSA).
Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages
Carlos Mullov (Karlsruhe Institute of Technology), Alexander Waibel (Karlsruhe Institute of Technology)
CodeTransformerContrastive LearningText
π― What it does: Proposes a decoupled lexical learning approach for multilingual NMT, first training word vectors on monolingual corpora and performing cross-lingual alignment, then freezing the word embeddings in the encoder to achieve zero-shot translation for unseen languages and apply it to unsupervised MT.
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Damai Dai (Peking University), Wenfeng Liang (Peking University)
CodeTransformerLarge Language ModelMixture of ExpertsTextBenchmark
π― What it does: Propose the DeepSeekMoE model, achieving efficient expert specialization through fine-grained expert partitioning and shared expert isolation, thereby enhancing the performance of MoE language models while maintaining the parameter scale unchanged.
Detection-Correction Structure via General Language Model for Grammatical Error Correction
Wei Li (Peking University), Houfeng Wang (Peking University)
CodeTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: Proposed DeCoGLM, an integrated detection-correction framework based on GLM, which can perform error detection and local correction within a single model.
CodeTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
π― What it does: Constructed a large-scale NLP benchmark across dialects and variantsβDIALECTBENCH, covering 40 language families, 281 dialects/variants, and 10 text tasks.
Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillation
Aiwei Liu (Tsinghua University), Lijie Wen (Tsinghua University)
CodeReinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningPrompt EngineeringContrastive LearningText
π― What it does: Propose a fully automatic alignment method for LLMs called DLMA, which does not require human-annotated preference data. It generates response pairs using contrastive prompts and aligns them through probabilistic comparison of self-assessment scores.
Discursive Socratic Questioning: Evaluating the Faithfulness of Language Modelsβ Understanding of Discourse Relations
Yisong Miao (National University of Singapore), Min-Yen Kan (National University of Singapore)
CodeExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
π― What it does: Propose the DISQ (Discursive Socratic Questioning) method, which evaluates the faithfulness of large language models (LLMs) in understanding discourse relations through a series of question-and-answer interactions; and systematically assess the discourse reasoning capabilities of multiple LLMs based on this framework.
Disentangled Learning with Synthetic Parallel Data for Text Style Transfer
Jingxuan Han (University of Science and Technology of China), Zhendong Mao (University of Science and Technology of China)
CodeGenerationData SynthesisTransformerLarge Language ModelContrastive LearningTextChain-of-Thought
π― What it does: Propose the DisenTrans framework, achieving text style transfer through attribute/content splitting; generate and filter high-quality parallel data using Chain-of-Thought (CoT), followed by training with contrastive and seq2seq losses.
π― What it does: Investigate whether the Functional Distributional Semantics (FDS) model can learn hierarchical relationships (hypernyms) from corpora and experimentally verify its performance on synthetic and real-world corpora.
CodeExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelText
π― What it does: Investigates whether Llama-2 uses English as an implicit 'pivot' language for non-English inputs, and analyzes the evolution of language preferences in its hidden representations across layers.
π― What it does: Constructed a benchmark named DOCMATH-EVAL to evaluate the numerical reasoning capabilities of large language models in understanding and analyzing long professional documents with mixed text and tables.
Document-level Claim Extraction and Decontextualisation for Fact-Checking
Zhenyun Deng (University of Cambridge), Andreas Vlachos (University of Cambridge)
CodeGenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
π― What it does: This paper proposes a document-level claim extraction and decontextualization framework that can identify central claims from multi-sentence documents and rewrite them into independently understandable sentences;
DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning
Yejie Wang (Beijing University of Posts and Telecommunications), Xunliang Cai (Meituan)
CodeAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: Proposes DolphCoder, an instruction-tuning model that generates diverse responses through multiple system prompts and combines code evaluation objectives to enhance code generation capabilities.
Donβt Buy it! Reassessing the Ad Understanding Abilities of Contrastive Multimodal Models
Anna Bavaresco (University of Amsterdam), Raquel FernΓ‘ndez (University of Amsterdam)
CodeRetrievalRecommendation SystemData-Centric LearningVision Language ModelContrastive LearningMultimodalityBenchmark
π― What it does: This paper re-examines the advertising understanding task, finding that existing evaluation settings can be exploited by models using anchoring cues between text and images to achieve high scores, and proposes a new adversarial test set called TRADE.
Donβt Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
Shangbin Feng (University of Washington), Yulia Tsvetkov (University of Washington)
CodeTransformerLarge Language ModelAgentic AIPrompt EngineeringText
π― What it does: Proposed and implemented an LLM 'give-up' mechanism to refuse answering when knowledge gaps occur, evaluated on multiple knowledge-intensive QA tasks.
DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models
Weihang Su (Tsinghua University), Yiqun Liu (Tsinghua University)
CodeGenerationRetrievalTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
π― What it does: Proposed a dynamic retrieval-augmented generation framework called DRAGIN, which can determine when and what to retrieve during text generation by large language models based on real-time information needs;
DUAL-REFLECT: Enhancing Large Language Models for Reflective Translation through Dual Learning Feedback Mechanisms
Andong Chen (Harbin Institute Of Technology), Min Zhang (Harbin Institute Of Technology)
CodeGenerationLarge Language ModelText
π― What it does: Proposed and implemented the DUAL-REFLECT framework, leveraging large language models and bidirectional learning (back-translation) loops to achieve self-reflection and improve translation quality.
DynaSemble: Dynamic Ensembling of Textual and Structure-Based Models for Knowledge Graph Completion
Ananjan Nandi (Indian Institute of Technology Delhi), Mausam .
CodeGraph Neural NetworkGraph
π― What it does: Propose a dynamic weighted ensemble method called DynaSemble, which utilizes the score distributions of text models and structural models to predict missing links in knowledge graphs.
EFSA: Towards Event-Level Financial Sentiment Analysis
Tianyu Chen (Key Laboratory of AI Safety, Chinese Academy of Sciences), Xiang Ao (Key Laboratory of AI Safety, Chinese Academy of Sciences)
CodeClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkFinance RelatedChain-of-Thought
π― What it does: This paper proposes an event-level task for financial sentiment analysis (EFSA), converting event extraction into a classification task, and constructs a predictive framework containing five components: company, industry, coarse and fine-grained event categories, and sentiment polarity, while providing a corresponding large-scale Chinese dataset.
π― What it does: Proposed a new Transformer architecture called Enhanced Interactive Transformer (EIT), introducing multi-to-multi mapping (M2M), as well as internal subspace interaction (ISI) and cross-subspace interaction (CSI) within its multi-head self-attention mechanism to achieve collaborative consistency among attention heads.
End-to-end Learning of Logical Rules for Enhancing Document-level Relation Extraction
Kunxun Qi (Sun Yat-sen University), Hai Wan (Sun Yat-sen University)
CodeExplainability and InterpretabilityRepresentation LearningTransformerSupervised Fine-TuningTextBenchmark
π― What it does: Proposed an end-to-end framework called JMRL for learning logical rules, jointly training a document-level relation extraction model and a rule reasoning module, enabling rule learning and extraction tasks to be optimized simultaneously within the same training process.