ACL 2024 Papers with AI Summaries
Annual Meeting of the Association for Computational Linguistics · 940 papers
→ ACL 2024 papers with code (356)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 940 ACL 2024 papers by keyword, author or institution —
free trial on arXivSub.
\inftyBench: Extending Long Context Evaluation Beyond 100K Tokens
Xinrong Zhang (Tsinghua University), Maosong Sun (Tsinghua University)
Data SynthesisRetrievalAI Code AssistantTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed the ∞BENCH benchmark, covering multi-domain bilingual long-context tasks with an average of over 100K tokens.
A Causal Approach for Counterfactual Reasoning in Narratives
Feiteng Mu (Hong Kong Polytechnic University), Wenjie Li (Hong Kong Polytechnic University)
GenerationData SynthesisTransformerLarge Language ModelReinforcement LearningAuto EncoderContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Propose a causal framework based on Variational Autoencoder (VAE) to generate counterfactual stories consistent with given scenarios, enhanced by a pre-trained entailment classifier and external event commonsense knowledge.
A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
Alon Jacovi (Google Research), Mor Geva (Google Research)
Explainability and InterpretabilityTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Construct and release the REVEAL dataset to evaluate the relevance, attribution, and logical correctness of each step in the chain-of-thought (CoT) reasoning of language models, and assess existing automatic validation methods.
A Community-Centric Perspective for Characterizing and Detecting Anti-Asian Violence-Provoking Speech
Gaurav Verma (Georgia Institute of Technology), Srijan Kumar (Georgia Institute of Technology)
ClassificationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Construct a codebook from a community perspective for violent incitement, use this codebook to cluster and screen 42,000 Twitter posts, then generate 1,000 high-density samples through few-shot learning (PET). Utilize Asian community members to conduct multi-round voting-style annotation to obtain labeled data for violent incitement and hate speech. Subsequently, train and compare the performance of BERT/RoBERTa and LLMs (Mixtral) on tasks of hate speech and violent incitement detection.
A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques
Megh Thakkar (Mila - Quebec AI Institute), Sarath Chandar (Mila - Quebec AI Institute)
Computational EfficiencyReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Systematically experiment on the impact of parameter-efficient preference alignment techniques (LoRA/QLoRA) on LLaMA-1, Mistral-7b, and their instruction-tuned versions, exploring how three axes (dataset, alignment method, base model) affect downstream task performance.
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia
Giovanni Monea (EPFL), Robert West (EPFL)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper investigates the 'grounding' capability of large language models (LLMs) when facing contexts conflicting with their internal parameter knowledge. It constructs a novel counterfactual dataset, Fakepedia, benchmarks multiple LLMs, and reveals computational pattern differences between grounding and recall through an improved causal mediation analysis (Masked Grouped Causal Tracing, MGCT). It also demonstrates that computational graphs alone can distinguish grounded from non-grounded answers, achieving an accuracy of 92.8%.
A Joint Coreference-Aware Approach to Document-Level Target Sentiment Analysis
Hongjie Cai (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
ClassificationTransformerLarge Language ModelText
🎯 What it does: This paper proposes a joint coreference-aware document-level target sentiment analysis framework, which utilizes dual-path RoBERTa multi-task learning to simultaneously perform target extraction, sentiment classification, and hierarchical relationship inference, while enhancing model performance through coreference information.
A Modular Approach for Multimodal Summarization of TV Shows
Louis Mahon (University of Edinburgh), Mirella Lapata (University of Edinburgh)
GenerationTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Built a modular multi-modal long-form TV program summarization system, decomposing the task into five subtasks: scene detection, scene reordering, visual-to-text conversion, dialogue summarization, and high-level summary fusion.
A Multi-Task Embedder For Retrieval Augmented LLMs
Peitian Zhang (Renmin University of China), Jian-Yun Nie (Beijing Academy of Artificial Intelligence)
RetrievalKnowledge DistillationRepresentation LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Propose a unified embedding model called LLM-Embedder to support large language models (LLMs) in generating and retrieving embeddings across four retrieval-enhanced scenarios: knowledge retrieval, memory retrieval, example retrieval, and tool retrieval.
A Multidimensional Framework for Evaluating Lexical Semantic Change with Social Science Applications
Naomi Baes (University of Melbourne), Ekaterina Vylomova (University of Melbourne)
TransformerLarge Language ModelText
🎯 What it does: Proposed and implemented a three-dimensional semantic change evaluation framework (valence, breadth, intensity), and applied it to systematically analyze the semantic evolution of mental health and mental illness concepts in academic psychology and general American English corpora.
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Any Translation
Zhengrui Ma (Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences), Min Zhang (Soochow University)
GenerationTransformerGenerative Adversarial NetworkTextAudio
🎯 What it does: Proposes a non-autoregressive end-to-end simultaneous interpretation framework (NAST-S2x) that can simultaneously handle speech-to-text (Simul-S2T) and speech-to-speech (Simul-S2S) tasks, achieving real-time generation through a block-level encoder + block-level non-autoregressive decoder.
A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus
Eduard Poesina, Radu Tudor Ionescu
ClassificationData-Centric LearningTransformerTextBenchmark
🎯 What it does: Studied the natural language inference (NLI) task for the Romanian language, created the first RoNLI corpus, and designed a curriculum learning strategy based on data visualization methods.
A Sentiment Consolidation Framework for Meta-Review Generation
Miao Li (University of Melbourne), Eduard Hovy (University of Melbourne)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This study proposes a three-tier sentiment merging framework for automatically generating meta-reviews in academic peer review, and designs new prompting methods and evaluation metrics based on this framework;
A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts
Nafis Irtiza Tripto (Pennsylvania State University), Dongwon Lee (Pennsylvania State University)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper systematically evaluates the authorship attribution problem of large language models (LLM) on text after multi-round rewriting, exploring whether rewriting leads to changes in author identity;
A synthetic data approach for domain generalization of NLI models
Mohammad Javad Hosseini (Google Deepmind), Annie Louis (Google Deepmind)
ClassificationData SynthesisDomain AdaptationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a method to generate a synthetic NLI dataset (GNLI) using large language models, which is multi-domain, multi-length, and label-balanced, and trains NLI models with it to enhance cross-domain generalization capabilities.
A Unified Temporal Knowledge Graph Reasoning Model Towards Interpolation and Extrapolation
Kai Chen (National University of Defense Technology), Xin Song (National University of Defense Technology)
Explainability and InterpretabilityRepresentation LearningGraph Neural NetworkGraphTime SeriesSequential
🎯 What it does: Propose a unified temporal knowledge graph reasoning model called TPAR, which can simultaneously handle two types of reasoning tasks: interpolation and extrapolation.
ABEX: Data Augmentation for Low-Resource NLU via Expanding Abstract Descriptions
Sreyan Ghosh (University of Maryland), Dinesh Manocha (University of Maryland)
ClassificationRecognitionData SynthesisData-Centric LearningGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose the ABEX method, which first converts documents into concise abstract descriptions and then uses BART to expand these abstractions, generating diverse augmented samples.
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters
Li Lucy (Allen Institute for AI), Jesse Dodge (Allen Institute for AI)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Constructed the AboutMe dataset containing 10.3M website self-described pages, automatically extracted authors' thematic interests, personal/organizational identities, social roles, and geographic information, and evaluated the preferences and removal rates of ten 'quality' filters and English language identification (langID) filters when processing these pages.
AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimation
Zhaowei Wang (HKUST), Simon See (NVIDIA)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: Design and implement the ABSINSTRUCT framework, enhancing the abstract reasoning capabilities of large language models by incorporating detailed explanation trajectories and a suspicion estimator in instruction tuning.
Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space
Zongru Wu (Shanghai Jiao Tong University), Gongshen Liu (Shanghai Jiao Tong University)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a defense method called MuScleLoRA against backdoor attacks, which trains language models using frequency-domain multi-scale scaling and low-rank adapters (LoRA) to suppress the learning of low-frequency features from backdoor mappings, enhance the learning priority of clean mappings, and thus train a clean model on poisoned data.
ActionIE: Action Extraction from Scientific Literature with Programming Languages
Xianrui Zhong (University of Illinois Urbana-Champaign), Jiawei Han (University of Illinois Urbana-Champaign)
GenerationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This study proposes the ActionIE method, which utilizes large language models to convert experimental procedures in chemical literature into executable Python code, achieving structured extraction of experimental steps.
Active Prompting with Chain-of-Thought for Large Language Models
Shizhe Diao (Hong Kong University of Science and Technology), Tong Zhang (University of Illinois Urbana Champaign)
Data-Centric LearningTransformerPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposed the Active-Prompt method, which actively selects the most valuable examples through uncertainty metrics and manually annotates them for chain-of-thought reasoning, thereby enhancing the reasoning performance of large models.
Advancement in Graph Understanding: A Multimodal Benchmark and Fine-Tuning of Vision-Language Models
Qihang Ai (Beijing Institute Of Technology), Shuming Shi (Tencent)
Supervised Fine-TuningVision Language ModelContrastive LearningMultimodalityGraphBenchmarkChain-of-Thought
🎯 What it does: Propose to use a visual language model (VLM) to generate image representations of graph data, and achieve graph understanding and reasoning through multimodal instruction following, constructing a bilingual (Chinese-English) multimodal graph dataset containing knowledge graphs, route maps, mind maps, flowcharts, and Gantt charts.
Advancing Abductive Reasoning in Knowledge Graphs through Complex Logical Hypothesis Generation
Jiaxin Bai (Hong Kong University of Science and Technology), Yangqiu Song (Hong Kong University of Science and Technology)
GenerationOptimizationExplainability and InterpretabilityTransformerReinforcement LearningGraph
🎯 What it does: Studied the task of abductive reasoning on knowledge graphs, proposing a method to generate complex logical hypotheses from given observations, and enhancing the interpretability of hypotheses through reinforcement learning.
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations
Guan-Ting Lin (National Taiwan University), Hung-yi Lee (National Taiwan University)
GenerationData SynthesisTransformerLarge Language ModelTextMultimodalityAudio
🎯 What it does: Developed a multimodal large model called Spoken-LLM, capable of generating corresponding voice responses based on different speaking styles for the same sentence.
Advancing Parameter Efficiency in Fine-tuning via Representation Editing
Muling Wu (Fudan University), Xuanjing Huang (Fudan University)
Computational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes a new parameter-efficient fine-tuning method called RED, which modifies network representations by directly scaling and bias-editing the hidden representations of Transformers, rather than adjusting weights.
AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators
Jingwei Ni (ETH Zürich), Markus Leippold (University of Zürich)
ClassificationData-Centric LearningTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Propose the AFaCTA framework, which leverages large language models (LLMs) to assist in the annotation work for fact claim detection.
AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
Zeyu Liu (University of Southern California), Peter Beerel (University of Southern California)
Computational EfficiencyKnowledge DistillationTransformerSupervised Fine-TuningText
🎯 What it does: Propose the AFLoRA method, which employs a trainable low-rank path and progressively freezes the projection matrix during fine-tuning;
AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts
Daniel Braun (University of Twente), Florian Matthes (Technical University of Munich)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Created the AGB-DE corpus, containing 3,764 German consumer contract clauses, manually annotated for clause legality (potentially invalid/valid) and topic labels, along with baseline evaluation.
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Da Yin (UCLA), Bill Yuchen Lin (Allen Institute for AI)
TransformerLarge Language ModelSupervised Fine-TuningAgentic AIPrompt EngineeringTextMultimodality
🎯 What it does: Developed an open-source, unified, and modular language agent framework called LUMOS, utilizing the LLAMA-2 7B/13B model and trained with unified annotations to support multi-task interaction;
Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization
Wenqi Zhang (Zhejiang University), Weiming Lu (Zhejiang University)
OptimizationTransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringWorld ModelTextSequential
🎯 What it does: Propose Agent-Pro, a self-evolving agent that leverages large language models to learn and improve behavioral policies in interactive environments through dynamic belief updates, strategy-level reflection, and DFS optimization;
AGR: Reinforced Causal Agent-Guided Self-explaining Rationalization
Yunxiao Zhao (Shanxi University), Ru Li (Shanxi University)
ClassificationExplainability and InterpretabilityRecurrent Neural NetworkReinforcement LearningAgentic AIText
🎯 What it does: Propose a selective explainability method AGR based on reinforced causal agents, which uses causal intervention and reinforcement learning to guide the generator to gradually select more effective rationales during training.
AI ‘News’ Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italian
Giovanni Puccetti (Istituto di Scienza e Tecnologia dell'Informazione A Faedo), Andrea Esuli (Istituto di Scienza e Tecnologia dell'Informazione A Faedo)
GenerationData SynthesisAnomaly DetectionTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Fine-tuning Llama (and its subsequent models) using only about 40,000 Italian news articles can generate news content nearly indistinguishable from human writing, forming so-called 'content farms'.
AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension
Qian Yang (Zhejiang University), Jingren Zhou (Alibaba Group)
Large Language ModelTextMultimodalityBenchmarkAudio
🎯 What it does: Propose AIR-Bench, the first generative evaluation benchmark for large-scale audio-lingual models, containing 19 multiple-choice tasks and over 2000 open-ended questions, covering speech, natural sounds, and music, and introducing volume and time-shifted mixing strategies.
AlignBench: Benchmarking Chinese Alignment of Large Language Models
Xiao Liu (Tsinghua University), Jie Tang (Tsinghua University)
TransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Proposed ALIGNBENCH, a Chinese LLM alignment evaluation benchmark, covering 8 categories with 683 real-world scenario queries, and providing human-machine verified reference answers and evidence.
Aligning Large Language Models by On-Policy Self-Judgment
Sangkyu Lee (Yonsei University), Youngjae Yu (Yonsei University)
Reinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes the SELF-JUDGE framework, which uses a single model to simultaneously serve as both the policy and the judge, achieving self-alignment without an additional reward model through Judge-Augmented Supervised Fine-Tuning (JSFT).
Aligning Large Language Models for Controllable Recommendations
Wensheng Lu (Shenzhen University), Xing Xie (Microsoft Research Asia)
Recommendation SystemTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTabularSequential
🎯 What it does: This paper connects large language models (LLMs) as controllable recommendation agents through a two-stage fine-tuning approach (supervised learning + reinforcement learning), supporting implicit, item-level, and list-level instructions.
Aligning Large Language Models via Fine-grained Supervision
Dehong Xu (UCLA), Jaeyoung Do (Amazon)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper proposes a fine-grained reinforcement learning with human feedback (Fine-grained RLHF) framework, which generates fine-grained annotated data by letting humans or large language models make minimal edits to low-quality answers; and trains a token-level reward model based on this, further used for PPO optimization;
Aligning Large Language Models with Human Preferences through Representation Engineering
Wenhao Liu (Fudan University), Xuanjing Huang (Fudan University)
Representation LearningReinforcement Learning from Human FeedbackLarge Language ModelSupervised Fine-TuningContrastive LearningText
🎯 What it does: Proposes a method called RAHF for aligning large language models by leveraging differences in internal representations caused by contrasting human preferences.
American Sign Language Handshapes Reflect Pressures for Communicative Efficiency
Kayo Yin (University of California Berkeley), Dan Klein (University of California Berkeley)
Pose EstimationTextPoint Cloud
🎯 What it does: Investigate the communication efficiency of American Sign Language (ASL) handshapes, quantify articulatory and perceptual burden, and explore their relationship with the usage frequency of handshapes in native vocabulary versus borrowed (English) vocabulary.
An Effective Pronunciation Assessment Approach Leveraging Hierarchical Transformers and Pre-training Strategies
Bi-Cheng Yan (National Taiwan Normal University), Berlin Chen (Chunghwa Telecom Co., Ltd)
ClassificationTransformerAudio
🎯 What it does: Propose HierTFR, a hierarchical Transformer framework for multi-faceted, multi-grained automatic pronunciation assessment;
An Empirical Analysis on Large Language Models in Debate Evaluation
Xinyi Liu (University of Rochester), Hangfeng He (University of Rochester)
Large Language ModelText
🎯 What it does: Investigated the capabilities and biases of large language models (LLMs) in debate evaluation and systematically assessed their performance.
An Entropy-based Text Watermarking Detection Method
Yijian Lu (Chinese University of Hong Kong), Irwin King (Chinese University of Hong Kong)
Anomaly DetectionText
🎯 What it does: This paper proposes a text watermark detection method based on token entropy (EWD), which significantly improves detection accuracy in low-entropy scenarios by assigning different detection weights to tokens with varying entropy.
An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
Ziwei Chai (Zhejiang University), Yang Yang (Zhejiang University)
Large Language ModelMixture of ExpertsText
🎯 What it does: Built a unified expert LLM collaboration framework called Expert-Token-Routing, which encodes expert LLMs into the meta LLM's vocabulary through dedicated tokens, enabling seamless collaboration and routing among multiple expert LLMs.
An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation
Kun Zhu (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
GenerationRetrievalCompressionTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: Propose introducing the information bottleneck (IB) theory into noise filtering within retrieval-augmented generation, creating a noise filter that simultaneously balances information compression and maintains output relevance.
An Information-Theoretic Approach to Analyze NLP Classification Tasks
Luran Wang (University of Cambridge), Vatsal Raina (University of Cambridge)
ClassificationTransformerLarge Language ModelText
🎯 What it does: Propose an information-theoretic framework that quantifies the impact of input elements (semantic and linguistic realization) on the output distribution in text classification tasks using mutual information.
An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs
Daking Rai (George Mason University), Ziyu Yao (George Mason University)
Explainability and InterpretabilityTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Studied the neuron activation patterns of LLMs in chain-of-thought (CoT) arithmetic reasoning, using GPT-4 to automatically discover FF layer neurons related to arithmetic reasoning, and explained previous CoT experimental observations through activation analysis.
An Iterative Associative Memory Model for Empathetic Response Generation
Zhou Yang (Fuzhou University), Xiangwen Liao (Fuzhou University)
GenerationTransformerTextBenchmark
🎯 What it does: Propose the Iterative Associative Memory Model (IAMM), which iteratively captures associative words in dialogues through a second-order interaction attention mechanism on explicit and implicit information, thereby generating more empathetic and relevant responses.
An Open Multilingual System for Scoring Readability of Wikipedia
Mykola Trokhymovych (Pompeu Fabra University), Martin Gerlach (Wikimedia Foundation)
Data-Centric LearningTransformerLarge Language ModelContrastive LearningTextBenchmark
🎯 What it does: Developed a multilingual model for automatically assessing the readability of Wikipedia articles, providing a directly usable tool through a public API.
ANAH: Analytical Annotation of Hallucinations in Large Language Models
Ziwei Ji (Shanghai AI Laboratory), Kai Chen (Shanghai AI Laboratory)
ClassificationAnomaly DetectionTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Constructed a bilingual (English-Chinese) large corpus ANAH, focusing on fine-grained annotation for each sentence in the answers generated by large language models for question-answering tasks: retrieving corresponding reference passages, determining hallucination types (no hallucination/contradictory hallucination/unverifiable hallucination/no factual content), and providing correction suggestions; subsequently trained and evaluated generative and discriminative hallucination annotators using this dataset.
ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base
Siyu Yuan (Fudan University), Deqing Yang (Fudan University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningTextGraphBenchmark
🎯 What it does: Constructed a large-scale analogy knowledge base ANALOGYKB with over a million entries, leveraging LLMs to extract analogous relationships of both identical and similar types from Wikidata and ConceptNet.
Analysing The Impact of Sequence Composition on Language Model Pre-Training
Yu Zhao (University of Edinburgh), Pasquale Minervini (University of Edinburgh)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningData-Centric LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Analyze the impact of pre-training sequence composition on language models, compare different packing and masking strategies, and propose the BM25Chunk retrieval-based packing method.
Analysis of Multi-Source Language Training in Cross-Lingual Transfer
Seonghoon Lim (Hanyang University), Taeuk Kim (Hanyang University)
Domain AdaptationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Investigate the effectiveness and mechanisms of multi-source language training (MSLT) in cross-lingual transfer, analyze the impact of language combinations and the number of source languages on performance, and propose a heuristic for selecting source languages based on linguistic diversity (e.g., Lang2Vec, writing system diversity).
Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trends
Sanjana Ramprasad (Northeastern University), Zachary Lipton (Abridge AI)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Conduct a fine-grained evaluation of the reliability of LLMs in dialogue summarization, propose the 'Circumstantial Inference' error type, and build a fine-grained annotation and error classification system.
Analyzing Semantic Change through Lexical Replacements
Francesco Periti, Nina Tahmasebi
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper constructs lexical substitutions with different semantic relationships (synonyms, antonyms, hypernyms, random words) to simulate semantic changes and observe the embedding distance variations of pre-trained language models in different contexts, thereby quantifying the model's tension towards semantic evolution, and proposes an interpretable method for semantic change detection based on this.
Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding
Zhihan Zhang (Fudan University), Tat-Seng Chua (National University of Singapore)
TransformerLarge Language ModelPrompt EngineeringTextSequentialBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose a time-complex event (TCE) analysis framework based on large language models (LLMs) and construct the TCELongBench benchmark dataset.
Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution
Flor Miriam Plaza-del-Arco (Bocconi University), Dirk Hovy (Bocconi University)
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Investigate gender stereotypes in large language models during emotion attribution tasks, revealing their gendered predictions about emotions
Annotating FrameNet via Structure-Conditioned Language Generation
Xinyue Cui (University of Southern California), Swabha Swayamdipta (University of Southern California)
ClassificationGenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This study proposes a structure-based conditional language generation framework that generates new semantically structured sentences using LLM on FrameNet to fill in unannotated lexical units.
Answer is All You Need: Instruction-following Text Embedding via Answering the Question
Letian Peng (University of California, San Diego), Jingbo Shang (University of California, San Diego)
Representation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes INBEDDER, a text embedder that achieves instruction following by generating answers;
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
Jun Zhan (Fudan University), Xipeng Qiu (Fudan University)
GenerationData SynthesisTransformerLarge Language ModelMultimodality
🎯 What it does: Built AnyGPT, a general large language model that achieves arbitrary multimodal input and output through discrete representations, and realizes arbitrary multimodal interaction using synthesized AnyInstruct-108k dialogue data.
AoE: Angle-optimized Embeddings for Semantic Textual Similarity
Xianming Li (Hong Kong Polytechnic University), Jing Li (Hong Kong Polytechnic University)
RetrievalRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningText
🎯 What it does: This paper proposes the Angle-optimized Embedding (AoE) model, which splits text embeddings into real and imaginary parts, computes angle differences in complex space, and introduces an angle loss to address the gradient disappearance problem caused by the saturation region of cosine similarity.
API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs
Kinjal Basu (IBM Research), Luis Lastras
Data SynthesisTransformerLarge Language ModelSupervised Fine-TuningTextSequentialBenchmark
🎯 What it does: This paper proposes and constructs the API-BLEND dataset, providing multi-domain, real-world API call data for training and evaluating tool-enhanced large language models.
AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
Harsh Trivedi (Stony Brook University), Niranjan Balasubramanian (Stony Brook University)
AI Code AssistantLarge Language ModelAgentic AITextTabularBenchmark
🎯 What it does: Built the AppWorld Engine and AppWorld Benchmark, providing a controllable multi-app simulation environment and 750 complex interactive coding tasks.
Arabic Diacritics in the Wild: Exploiting Opportunities for Improved Diacritization
Salman Elgamal (New York University Abu Dhabi), Nizar Habash (New York University Abu Dhabi)
ClassificationRecognitionData-Centric LearningTransformerText
🎯 What it does: This paper analyzes naturally occurring partial diacritics (WildDiacs), constructs a new maximized diacritics dataset called Wild2MaxDiacs, and improves the analysis-disambiguation workflow of CAMeL Tools. By leveraging WildDiacs for re-ranking and contextual post-processing, high-quality re-diacritization across multiple corpora is achieved.
ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation
Chen Huang (Sichuan University), Jiancheng Lv (Sichuan University)
Data-Centric LearningTransformerText
🎯 What it does: Propose the ARAIDA method in interactive data annotation scenarios, utilizing KNN models and error-aware ensemble strategies to dynamically compensate for errors in traditional labeling models under limited labeled data, thereby reducing the workload of manual error correction.
ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models
Hojae Han (Seoul National University), Seung-won Hwang (Seoul National University)
AI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Propose the ARCHCODE framework, which leverages context learning of large language models to automatically extract and infer functional (FR) and non-functional requirements (NFR) from natural language requirements, using them as conditions to generate code and corresponding test cases, followed by filtering and sorting the generated code through executing the test cases;
Are AI-Generated Text Detectors Robust to Adversarial Perturbations?
Guanhua Huang (University of Science and Technology of China), Zhouwang Yang (University of Science and Technology of China)
Anomaly DetectionAdversarial AttackTransformerLarge Language ModelAuto EncoderContrastive LearningTextBenchmark
🎯 What it does: This paper proposes a novel AI-generated text detector called Siamese Calibrated Reconstruction Network (SCRN), aiming to enhance robustness against word/character-level adversarial perturbations.
Are Emergent Abilities in Large Language Models just In-Context Learning?
Sheng Lu (Technical University of Darmstadt), Iryna Gurevych (Technical University of Darmstadt)
Explainability and InterpretabilityTransformerSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Studied the 'emergent abilities' of large language models (LLMs) and verified whether these abilities are inherent to the models or driven by in-context learning (ICL), model memory, and linguistic knowledge through controlling ICL and instruction tuning.
Are LLM-based Evaluators Confusing NLG Quality Criteria?
Xinyu Hu (Wangxuan Institute of Computer Technology, Peking University), Xiaojun Wan (Wangxuan Institute of Computer Technology, Peking University)
GenerationExplainability and InterpretabilityAdversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Conduct fine-grained perturbation attack testing on the reliability of large language models (LLMs) in natural language generation (NLG) evaluation, revealing their confusion and over-sensitivity in distinguishing different evaluation dimensions (e.g., fluency, consistency, factual accuracy).
Are LLMs classical or nonmonotonic reasoners? Lessons from generics
Alina Leidinger (University of Amsterdam), Ekaterina Shutova (University of Amsterdam)
TransformerPrompt EngineeringTextBenchmark
🎯 What it does: This paper systematically evaluates the performance of seven LLMs on non-monotonic reasoning tasks by combining general statements with instantiations/exceptions.
Argument Mining in Data Scarce Settings: Cross-lingual Transfer and Few-shot Techniques
Anar Yeginbergen (University of Basque Country), Rodrigo Agerri (University of Basque Country)
Domain AdaptationData-Centric LearningTransformerSupervised Fine-TuningPrompt EngineeringTextBiomedical Data
🎯 What it does: Address the data scarcity problem in argument mining within medical text, comparing cross-lingual transfer, data transfer, and few-shot learning methods.
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews
Mike D’Arcy, Doug Downey (Allen Institute for AI)
ClassificationGenerationTransformerLarge Language ModelText
🎯 What it does: Propose and evaluate two new tasks: comment-edit alignment of scientific papers based on peer reviews, and edit generation based on peer reviews.
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
Haoxiang Wang (University of Illinois Urbana-Champaign), Tong Zhang (University of Illinois Urbana-Champaign)
Reinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: Developed a Directional Preference Alignment (DPA) framework that aligns LLMs with multi-dimensional user preferences using multi-objective rewards and user preference direction vectors, achieving fine-grained generation control through arithmetic manipulation.
ARL2: Aligning Retrievers with Black-box Large Language Models via Self-guided Adaptive Relevance Labeling
LingXi Zhang, Chao Zhang (Georgia Institute of Technology)
RetrievalRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes the ARL2 method, which enhances the accuracy of retrieval-augmented generation models by adaptively training the retriever through self-labeled retrieval relevance generated by large language models (LLMs).
Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?
Nishant Balepur (University of Maryland), Rachel Rudinger (University of Maryland)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Investigates the ability of large language models (LLMs) to complete multiple-choice questions (MCQA) when only the answer choices are provided without the corresponding question, and systematically evaluates their performance, potential shortcuts (such as memorization, selection dynamics, and question inference), and implications for dataset quality.
ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs
Fengqing Jiang (University of Washington), Radha Poovendran (University of Washington)
Safty and PrivacyAdversarial AttackLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This study investigates ASCII art-based jailbreak attacks, proposing the ArtPrompt method that exploits LLMs' insufficient recognition of ASCII art to bypass security filters.
Ask Again, Then Fail: Large Language Models’ Vacillations in Judgment
Qiming Xie (Nanjing University of Science and Technology), Rui Xia (Nanjing University of Science and Technology)
Data SynthesisOptimizationTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper first proposes the 'Follow-up Questioning Mechanism' (Follow-up Questioning Mechanism) and two consistency evaluation metrics (Modification and Modification Rate), systematically assessing the fluctuation of judgments of various large language models when facing subsequent questions; subsequently, to address this issue, two mitigation strategies are explored: 1) zero-shot prompting for closed-source models (Zero-shot CoT, EmotionPrompt, Few-shot Prompt); 2) the training framework UNWAVERING-FQ for open-source models, which includes data preparation, bias preference context distillation, and preference optimization based on DPO.
ATLAS: Improving Lay Summarisation with Attribute-based Control
Zhihao Zhang (Beijing University of Technology), Chenghua Lin (University of Manchester)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBiomedical Data
🎯 What it does: Propose the ATLAS model to achieve controllable generation of lay abstracts for scientific articles, using four attributes (length, readability, background information, lexical entropy) to adjust the 'understandability' of the abstracts.
Attribute First, then Generate: Locally-attributable Grounded Text Generation
Aviv Slobodkin (Bar Ilan University), Ido Dagan (Bar Ilan University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextChain-of-Thought
🎯 What it does: Propose a framework called 'attribute-first generation' that splits the generation process into content selection, sentence planning, and sentence-by-sentence generation, achieving synchronous generation of text and fine-grained references.
AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning
Shuofei Qiao (Zhejiang University), Huajun Chen (Zhejiang University)
TransformerLarge Language ModelTextMultimodality
🎯 What it does: Propose AUTOACT, an automated QA agent learning framework from scratch without requiring large-scale labeled data or closed-source models.
AutoDSL: Automated domain-specific language design for structural representation of procedures with constraints
Yu-Zhe Shi, Qining Wang (Peking University)
AI Code AssistantTransformerLarge Language ModelTextBiomedical Data
🎯 What it does: Automatically generate a domain-specific language (DSL) tailored for laboratory protocols to precisely constrain their structure and semantics, thereby assisting large language models (LLMs) in understanding and executing experimental protocols across five experimental fields.
Automated Justification Production for Claim Veracity in Fact Checking: A Survey on Architectures and Approaches
Islam Eldifrawi (Université de Sherbrooke), Amine Trabelsi (Université de Sherbrooke)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextMultimodalityReview/Survey PaperRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper reviews research on evidence explanation (proof) generation in automatic fact-checking, proposes a multi-dimensional classification framework, systematically compares different generation methods and pipeline architectures, and outlines future research directions.
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
Shivalika Singh (Cohere For AI Community), Sara Hooker (Cohere For AI)
Data-Centric LearningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed the Aya dataset and Aya Collection, containing 65 human-annotated instruction-completion pairs, 44 template datasets, 19 translation datasets, totaling 513 million instances, and open-sourced a multilingual evaluation suite.
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
Ahmet Üstün (Cohere For AI), Sara Hooker (Cohere For AI)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Built and released Aya, a multilingual instruction-tuned large language model (LLM) with 13B parameters, covering 101 languages (over 50% low-resource), along with diverse training mixes, evaluation suites, and safety mechanisms.
Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations
Gregor Geigle (WüNLP), Goran Glavaš (WüNLP)
ClassificationRetrievalTransformerPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodalityBenchmark
🎯 What it does: Created Babel-ImageNet, a multilingual zero-shot image classification benchmark that translates ImageNet class labels into 100 languages, and evaluated multilingual CLIP models.
Back to Basics: Revisiting REINFORCE-Style Optimization for Learning from Human Feedback in LLMs
Arash Ahmadian (Cohere For AI), Sara Hooker (Cohere For AI)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Explores and verifies whether simplified REINFORCE and its multi-sample variant RLOO can replace the commonly used PPO in large language model (LLM) reinforcement learning with human feedback (RLHF), further demonstrating that partial sequence modeling is unnecessary in RLHF scenarios.
BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents
Yifei Wang (Zhengzhou University), Shengsheng Qian (Zhengzhou University)
Adversarial AttackTransformerLarge Language ModelSupervised Fine-TuningAgentic AIText
🎯 What it does: Proposed a backdoor attack named BadAgent targeting large language model agents, demonstrating both active and passive triggering methods, which can implant backdoors during the fine-tuning phase and execute hidden operations after deployment.
BatchEval: Towards Human-like Text Evaluation
Peiwen Yuan (Beijing Institute of Technology), Kan Li (Beijing Institute of Technology)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Proposes the BATCHEVAL paradigm for batch evaluation of text quality, significantly improving the accuracy, robustness, and cost efficiency of LLM assessment through iterative heterogeneous batch composition, a two-stage analysis-scoring process, and decimal scoring.
BeamAggR: Beam Aggregation Reasoning over Multi-source Knowledge for Multi-hop Question Answering
Zheng Chu (Harbin Institute of Technology), Bing Qin (Harbin Institute of Technology)
RetrievalTransformerPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the BeamAggR framework, which decomposes multi-hop question answering into a tree structure and reduces reasoning errors while improving answer quality through bottom-up multi-source knowledge reasoning and beam search aggregation.
Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation
Tianqi Zhong (University of Science and Technology of China), Zhendong Mao (University of Science and Technology of China)
GenerationMeta LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the CompMCTG benchmark and the Meta-MCTG training framework to evaluate and enhance the compositional generalization capabilities of multi-attribute controllable text generation models.
Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations
Jiaxing Sun (State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University), Conghui He (Shanghai AI Laboratory)
Prompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Created and evaluated the Chinese common sense reasoning benchmark CHARM, covering both global and China-specific domains, and systematically assessed the reasoning and memory capabilities of 19 LLMs and 5 prompting strategies.
Benchmarking Data Science Agents
Yuge Zhang (Microsoft Research), Kan Ren (ShanghaiTech University)
AI Code AssistantLarge Language ModelAgentic AITextBenchmark
🎯 What it does: The paper proposes the DSEval evaluation framework and four benchmarks for comprehensive assessment of data science LLM agents.
Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation
Xunjian Yin (Wangxuan Institute of Computer Technology, Peking University), Xiaojun Wan (Wangxuan Institute of Computer Technology, Peking University)
Large Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: To address knowledge evaluation in large language models, this paper proposes an evaluation framework based on knowledge boundaries and designs the PGDC algorithm to automatically identify optimal prompts.
Beyond Memorization: The Challenge of Random Memory Access in Language Models
Tongyao Zhu (Sea AI Lab), Min Lin (Sea AI Lab)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Study the memory access patterns of language models and explore their performance in two scenarios: sequential access and random access.
Beyond Orthography: Automatic Recovery of Short Vowels and Dialectal Sounds in Arabic
Yassine El Kheir (Hamad Bin Khalifa University), Shammur Chowdhury
RecognitionTransformerAudio
🎯 What it does: Automatic recovery of short vowels and dialectal phonemes in Arabic dialect speech recognition
Beyond Recognising Entailment: Formalising Natural Language Inference from an Argumentative Perspective
Ameer Saadat-Yazdi (University of Edinburgh), Nadin Kökciyan (University of Edinburgh)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmark
🎯 What it does: Constructed the first multi-topic argumentation type dataset based on the periodic table argumentation framework, Kialo-PTA24, and developed the ArgNotator annotation tool. Subsequently, benchmark tests were conducted on pre-trained language models for two tasks: argumentation artifact classification and normalization.
Beyond Scaling: Predicting Patent Approval with Domain-specific Fine-grained Claim Dependency Graph
Xiaochen Gao (University of California San Diego), Jingbo Shang (University of California San Diego)
ClassificationGraph Neural NetworkLarge Language ModelPrompt EngineeringTextGraphChain-of-Thought
🎯 What it does: Propose a fine-grained dependency graph (FLAN Graph) based on patent claims, and represent it using graph neural networks (GNNs) to achieve patent approval prediction; meanwhile, systematically evaluate the impact of LLM scaling and prompting engineering on this task, proving that traditional LLMs cannot significantly improve performance.
Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation
Zdeněk Kasner (Charles University), Ondrej Dusek (Charles University)
GenerationTransformerLarge Language ModelPrompt EngineeringTextTabularBenchmark
🎯 What it does: This paper constructs the QUINTD tool on unlabeled real-world structured data and generates the QUINTD-1 dataset using this tool. It investigates the behavior of open-source large language models (LLMs) in data-to-text generation tasks, and evaluates semantic accuracy through human evaluation and reference-free metrics based on GPT-4.
Bi-Directional Multi-Granularity Generation Framework for Knowledge Graph-to-Text with Large Language Model
Haowei Du (Peking University), Dongyan Zhao (Peking University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextGraph
🎯 What it does: Propose a bidirectional multi-granularity generation framework (BDMG), which first generates a subset of knowledge graph triplets at the sentence level and then aggregates them to produce complete text; meanwhile, introduce a reverse relation extraction task to enhance the accuracy of relation information.
BinaryAlign: Word Alignment as Binary Sequence Labeling
Gaetan Latouche, Benjamin Swanson
ClassificationRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose BinaryAlign, redefining the word alignment task as a binary sequence labeling problem for each word pair, and uniformly handling high- and low-resource language pairs.